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ABSTRACT 


With  military  systems  becoming  increasingly  connected  and  more  agile  to  handle 
multiple  scenarios  through  technological  advances,  the  challenge  remains  to  design  a 
command  and  control  (C2)  system  that  fulfills  the  rising  user  expectation  given  the 
constraints  enforced  in  the  land  battlefield. 

While  there  is  always  ongoing  effort  to  narrow  the  gap  between  users’  expectation 
and  constraints  on  the  land  battlefield  through  integrating  more  sensors,  there  is  a  need  to 
manage  the  users’  expectation  to  ensure  the  effective  use  of  the  system  for  their 
operations  through  constant  evaluation  of  the  measures  of  effectiveness  (MOE). 

The  Langford  nine-step  methodology  provides  a  repeatable  process  to 
successfully  develop  twelve  pairs  of  meaningful  MOEs  using  the  integrative  framework 
to  evaluate  the  effectiveness  of  using  a  deployed  C2  system  in  the  land  battlefield.  The 
integrative  framework  provides  the  comprehensive  guidelines  to  develop  the  MOE  with 
objective  values  and  subjective  criteria.  The  nine-step  method’s  repeatability  facilitates 
the  evaluation  of  the  effectiveness  of  the  C2  for  each  system  refinement 
(firmware/software/system  upgrade)  or  new  requirements  (addition  of  sensor/ 
communication  system/protocol).  Each  evaluation  provides  opportunities  for  the  system, 
process,  and  organization  to  improve. 
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EXECUTIVE  SUMMARY 


With  advancement  in  technologies  such  as  platfonn  onboard  sensors,  combat 
communication  systems,  computing  power  and  human  system  integration  devices  to 
achieve  wider  coverage,  higher  data  bandwidth,  faster  system  response,  and  improved 
user  experience,  there  are  increasing  requirements  to  integrate  more  subsystems  to  the 
command  and  control  (C2)  system  to  further  enhance  the  situational  awareness  in  the 
network-centric  environment  and  improve  critical  decision  making  in  the  land  battlefield. 

However,  given  the  constraints  and  challenges  in  the  land  battlefield,  such  as 
terrain,  canopy,  and  equipment  life  cycle,  there  is  a  need  to  assess  the  benefits  and 
limitations  of  integrating  to  ensure  the  best  bang  for  the  buck.  Therefore,  there  is  a  need 
to  constantly  evaluate  the  measures  of  effectiveness  (MOE)  of  using  the  deployed  C2 
system  to  better  manage  users’  expectations  in  terms  of  additional  requirements  to 
integrate  better  sensor  equipment  to  enhance  situational  awareness  in  the  land  battlefield 
for  faster  and  quality  decision-making. 

Langford  (2014)  explains  “the  what,”  “the  when”  and  “the  how”  with  regards  to 
MOE.  “The  what”  must  be  interpreted  within  a  particular  context,  regardless  of  type. 
Without  boundaries,  it  is  uncertain  what  dimensionality  means  in  terms  of  effectiveness, 
let  alone  how  it  can  be  compared  or  how  meaningful  such  a  comparison  might  be.  “The 
how”  and  “the  when”  are  seemingly  attached  to  milestones  in  the  acquisition  parlance 
rather  than  to  the  merits  of  any  theoretical  foundation.  Langford  (2014,  8-12)  proposes 
developing  MOEs  using  an  integrative  framework. 

The  integrative  framework  is  used  to  develop  12  pairs  of  MOEs  to  evaluate  the 
effectiveness  of  using  the  deployed  C2  system  in  the  battlefield.  The  integrative 
framework  provides  the  comprehensive  guidelines  to  develop  the  MOEs  with  objective 
values  and  subjective  criteria.  The  nine-step  method’s  repeatability  facilitates  the 
evaluation  of  the  effectiveness  of  the  C2  for  each  system  refinement 
(firmware/software/system  upgrade)  or  new  requirements  (addition  of 
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sensor/communication  system/protocol).  Each  evaluation  should  indicate  opportunities 
for  the  system,  process,  and  organization  to  improve. 

The  developed  12  pairs  of  C2  MOEs  allow  stakeholders  to  evaluate  the  system 
and  the  use  of  the  integrative  framework  to  produce  a  repeatable  MOE  that  provides  cost 
saving  opportunities  for  system  refinement  and  new  iterations  of  system  development. 
The  12  pairs  of  the  C2  MOEs  are  used  to  evaluate  the  effectiveness  of  deploying  the 
Battlefield  Management  System  (BMS)  in  the  land  platfonn. 
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I.  INTRODUCTION 


With  the  rapid  improvement  in  technology  in  recent  years,  we  are  seeing  that  land 
platform  systems  are  becoming  more  precise  and  accurate.  Onboard  sensors,  more 
flexible  communication  systems,  advanced  computing  power,  and  more  robust  human 
system  integration  devices  achieve  wider  coverage,  higher  data  bandwidth,  faster  system 
response,  and  improved  user  experience.  These  technological  advancements  lead  to 
stakeholders’  interest  in  harnessing  the  capability  into  the  command  and  control  (C2) 
system  for  better  situational  awareness  and  decision-making.  This  interest  further 
translates  into  requirement  for  these  subsystems  integration  into  the  C2  system. 

To  manage  stakeholders’  expectations,  there  is  a  need  to  evaluate  the 
effectiveness  of  the  deployed  C2  system  having  implemented  with  the  new  requirements. 
The  measures  of  effectiveness  (MOEs)  of  using  the  deployed  C2  system  are  developed 
using  an  integrative  framework  to  evaluate  the  system’s  fitness-for-purpose. 

A.  USER  EXPECTATION  OF  COMMAND  AND  CONTROL  SYSTEMS  IN 
THE  LAND  BATTLEFIELD 

The  changes  that  have  taken  place  over  the  years  in  military  affairs  have  affected 
the  end-users’  expectation  of  using  the  C2  system  in  the  land  battlefield.  In  the  20th 
century,  the  C2  system  was  described  as  follows: 

C2  System  consists  of  a  collection  of  denotable  things  that  are  used  to 
perform  the  C2  function:  physical  elements  (e.g.,  equipment,  such  as 
transmitters  and  receivers,  computers,  a  signal  book,  status  boards  and 
other  decision  support  hardware,  code  breaking  facilities,  and  signal 
flags);  human  elements  (e.g.,  communicators,  staffs,  intelligence  analysts, 
and  the  commander  himself);  and  procedural  elements  (e.g.,  table  of 
organization  and  command  relationships,  the  content  of  a  signal  book,  and 
the  content  of  a  manual  of  training  or  doctrine).  This  collection  of  things  is 
presumed  in  most  usage  to  be  integrated.  The  purpose  of  the  C2  system  is 
to  facilitate  the  C2  process.  (Hughes  1989) 

However,  traditional  military  missions  have  evolved  over  the  years,  as  Alberts 
and  Hayes  (2006)  described: 
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Today’s  missions  differ  from  traditional  military  missions,  not  just  at  the 
margins,  but  qualitatively.  Today’s  missions  are  simultaneously  more 
complex  and  more  dynamic,  requiring  the  collective  capabilities  and 
efforts  of  many  organizations  in  order  to  succeed.  This  requirement  for 
assembling  a  diverse  set  of  capabilities  and  organizations  into  an  effective 
coalition  is  accompanied  by  shrinking  windows  of  response  opportunity. 
Traditional  approaches  to  C2  are  not  up  to  the  challenge.  Simply  stated, 
they  lack  the  agility  required  in  the  2 1  st  century. 

As  a  result  of  technological  breakthroughs  in  research  and  development, 
increasing  asymmetrical  challenges,  the  transition  from  stand-alone  systems  to 
increasingly  interconnected  systems  (e.g.,  the  emergence  of  system  of  systems  (SoS) 
ideology  and  the  network-centric  warfare  (NCW)),  user  expectations  on  the  C2  system 
has  inevitably  increased.  The  rising  user  expectations  have  led  to  an  increase  in 
requirements  to  integrate  the  C2  system  with  more  and  better  subsystems  to  provide 
enhanced  situational  awareness  for  better  decision-making.  In  addition,  C2  systems  are 
expected  to  be  more  agile  in  terms  of  system  configuration  and  deployment  time  to  meet 
the  faster  pace  of  mission  dynamism. 

Although  the  purpose  of  Command  and  Control  has  remained  unchanged 
since  the  earliest  military  forces  engaged  one  another,  the  way  we  have 
thought  about  Command  and  Control  and  the  means  by  which  the 
functions  of  Command  and  Control  have  been  accomplished  have  changed 
have  changed  significantly  over  the  course  of  history.  These  changes  have 
resulted  from  the  coevolution  of  Command  and  Control  Approaches  with 
technology,  the  nature  of  military  operations,  the  capabilities  of  forces, 
and  the  environments  in  which  militaries  operate.  (Alberts  and  Hayes 
2006) 

B.  PROBLEM  IDENTIFICATION 

With  the  increased  demand  to  integrate  more  subsystems  into  the  C2  system  to 
enhance  situational  awareness  and  faster  and  better  decision-making,  there  is  a  need  to 
evaluate  the  outcomes  (performance  and  experience)  in  order  to  determine  the  value 
(subjective  and  objective)  and  whether  the  effort  for  perfonning  the  integration  meets  the 
return  on  investment.  Thus,  MOEs  are  widely  applied  to  measure  and  evaluate  the 
performance  of  C2  systems  in  a  combat  context.  However,  the  challenge  lies  on  the 
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assumption  made  for  developing  the  MOEs,  which  sometimes  tend  to  be  short-sighted 
and  narrow. 

Langford  (2014)  describes  the  challenges  as  follows: 

Today,  the  selections  of  key  factors  in  detennining  the  measures  of 
effectiveness  are  made  without  a  consistent  approach  and  with  no 
underlying  theory  to  their  guide  selection.  The  current  ideas  and 
application  of  measures  of  effectiveness  incorrectly  emphasize  outcomes 
based  on  conjecture  that  some  aspect  of  a  performance  measure  or  metric 
is  an  indicator  of  utility  for  a  system’s  fitness.  (Langford  2014,  18) 

Bornman  (1993)  describes  the  problem  statement  for  developing  C2  MOE  as 
follows: 

The  lack  of  a  standardized  set  of  C2MOE  has  resulted  in  the  development 
of  measures  on  a  study-by-study  basis.  Most  previous  C2MOE  have  not 
clearly  linked  changes  in  C2  systems  or  doctrine  to  battle  outcome. 
C2MOE  have  tended  to  be  anecdotal  in  nature.  They  have  depended  upon 
a  high  degree  of  human  interaction,  and  thus,  have  been  prone  to 
inconsistency  in  either  measure  or  application.  In  addition,  because 
C2MOE  are  difficult  to  identify,  in  most  studies  and  evaluations  very  few 
are  used.  Evaluating  a  C2  system  with  just  one  of  two  MOE  can  limit  the 
focus  of  a  study  and  place  the  resulting  conclusions  in  jeopardy.  (Bornman 
1993,  15) 

Therefore,  it  is  important  to  address  the  challenge  of  MOE  development  so  that  a 
meaningful  assessment  can  be  performed  to  evaluate  the  effectiveness  of  using  the 
deployed  C2  system  in  the  land  battlefield. 

C.  SCOPE 

This  thesis  consists  of  four  parts:  first,  a  study  of  the  concept  of  C2  systems  in  the 
land  battlefield;  second,  a  study  of  the  various  concepts  of  developing  MOEs;  third,  an 
evaluation  of  MOEs  using  Langford’s  nine-step  method  and  integrative  framework;  and 
fourth,  a  conclusion  and  recommendation  for  future  work. 

D.  ORGANIZATION  OF  STUDY 

This  thesis  consists  of  six  chapters: 

Chapter  I:  Introduction 
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Chapter  II:  Background 

Chapter  III:  Literature  Review:  Measure  of  Effectiveness 
Chapter  IV:  Methodology  to  Develop  Measure  of  Effectiveness 
Chapter  V:  Development  of  Measure  of  Effectiveness 
Chapter  VI:  Conclusion 

This  chapter  identities  and  describes  the  challenges  of  MOE  development  to 
manage  the  user  expectation  of  using  the  deployed  C2  system  on  the  land  battlefield.  The 
next  chapter  provides  a  literature  review  on  the  background  of  the  C2  system  and  the 
rising  user  expectation  resulting  in  its  transition  from  stand-alone  system  to  the 
interconnected  system  of  systems. 
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II.  BACKGROUND 


This  chapter  provides  the  background  studies  of  the  C2  system  and  its  transition 
from  stand-alone  system  to  the  interconnected  system  of  systems.  It  is  important  to 
understand  the  C2  system  using  the  definition  of  command  and  control.  As  the 
expectation  of  the  stakeholder  is  to  stay  connected  in  the  battlefield  for  better  situational 
awareness  and  decision-making,  there  is  an  increasing  requirement  to  integrate  faster  and 
better  subsystems  (sensor,  communication  device  and  processor)  to  the  C2  system. 
However,  there  is  a  need  to  recognize  the  limitations  and  constraints  on  the  land 
battlefield  to  implement  these  requirements.  There  is  also  a  need  to  consider  the  system 
safety  aspect  while  evaluating  the  feasibility  of  the  solution. 

A.  COMMAND  AND  CONTROL 

Command  and  control  (C2)  is  defined  by  Joint  Chiefs  of  Staff  Publication  1  (JCS 
Pub  1)  as 

the  exercise  of  authority  and  direction  by  a  properly  designated 
commander  over  assigned  forces  in  the  accomplishment  of  his  mission. 
Command  and  control  functions  are  performed  through  an  arrangement  of 
personnel,  equipment,  communications,  facilities,  and  procedures  which 
are  employed  by  a  commander  in  planning,  directing,  coordinating,  and 
controlling  forces  and  operations  in  the  accomplishment  of  this  mission. 

The  C2  capability  can  be  further  decomposed  into  command  and  control  to 
distinguish  between  the  two  terms: 

Command  is  the  art  of  assigning  missions,  prioritizing  resources,  guiding 
and  directing  subordinates,  and  focusing  the  entire  division’s  energy  to 
accomplish  clear  objectives.  (Bornman  1993,  15). 

Control  is  the  science  of  defining  limits,  computing  requirements, 
allocating  resources,  prescribing  requirements  for  reports,  monitoring 
performance,  identifying  and  correcting  deviations  from  guidance,  and 
directing  subordinate  actions  to  accomplish  the  commander’s  intent. 
(Bornman  1993,  15). 

From  the  definitions,  C2  can  be  thought  of  as  having  six  components:  physical 
entities,  structure,  tasks  or  activities,  process,  function,  and  doctrinal  objective  or 
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mission,  as  shown  in  Figure  1  (Bornman  1993,  16).  As  shown  in  Figure  1,  the  lower 
components  are  derived  from  the  higher  components,  reflecting  the  policy  that  doctrine 
drives  the  development  of  training,  organizational  structures,  leaders,  and  material 
equipment  (Bornman  1993,  15-16).  The  evaluation  of  the  mission  success  uses  the 
bottom  up  approach  to  assess  the  effectiveness  of  each  component  and  its  influence  to  the 
next  higher  component. 


Figure  1.  Hierarchy  of  C2  Components  (from  Bornman  1993,  16) 


Doctrine  Devetopmert 
and  9*  Concept 
FtenapfranU  System 


Hierarchy  of  C2 


Objective  (Mission) 

Function 

Process 

Tasks/ Activities 

Structure 

Physical  Entities 

AnJyss  and  Evaluation 


( 1 )  Physical  Entities 

Bornman  (1993,  16)  refers  physical  entities  to  equipment,  software  facilities,  and 
people.  These  are  arranged  into  structures 

(2)  Structure 

Bornman  (1993,  16)  describes  the  structure  as  “the  arrangement  and 

interrelationships  of  physical  entities,  procedures,  protocols,  concepts  of  operation,  and 
information  patterns.” 
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(3)  Tasks/Activities 

Bornman  (1993,  16)  refers  tasks  or  activities  to  individual  and  collective  work  or 
actions  taken  by  the  entities  and  the  structure  as  a  part  of  a  process. 

(4)  Process 

Bornman  (1993,  16)  refers  process  to  the  arrangement  and  interrelationships  of 
tasks  or  activities  that  are  performed  to  fulfill  functions  defined  by  doctrine. 

(5)  Function 

Langford  (2013,  13)  describes  a  function  as  “a  set  of  actions  that  result  when  two 
objects  interact.  Processes  within  the  objects  enable  the  functions.  A  function  is  a  trait  of 
the  relation(s)  between  or  behavior(s)  of  objects  that  change  when  two  or  more  objects 
interact.  A  function  is  measureable  by  performance  of  the  bi-object’s  properties  or  traits, 
either  individually  or  pairwise.  Functions  arise  due  to  the  enactments  of  the  mechanisms 
of  at  least  two  objects.”  In  the  same  article,  Langford  explains  that  a  function  is 
“measured  by  performance  of  the  relations  between  the  interacting  objects  that  created 
the  function.”  A  function  is  characterized  by  the  perspective  of  a  stakeholder. 

A  formal  definition  of  function  from  Langford  also  include: 

•  Object  O  and  Object  P  create  a  function  f  if  and  only  if. 

•  Object  O  interacts  with  Object  P. 

•  Function  f  is  the  set  of  all  actions  that  are  a  consequence  of  Object  O  interacting  with 
Object  P. 

•  An  action  is  defined  as  the  release  or  receipt  of  EMMI  (Langford  2013,  308). 

•  EMMI  represents  energy,  matter,  material  wealth,  and  information.  EMMI  activates 
mechanisms. 

•  Object  O  is  changed  by  its  mechanism(s)  because  of  an  interaction  with  Object  P,  or 
Object  P  is  changed  by  its  mechanism(s)  because  of  an  interaction  with  Object  O,  or 
both  Object  O  and  Object  P  are  changed  by  their  respective  mechanism(s)  because  of 
an  interaction. 

•  Mechanism  is  set  of  rules  and  logic  constrained  by  context  and  environment  that 
govern  the  transformation  of  EMMI  by  objects. 
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•  A  change  is  defined  as  the  difference  between  an  object  before  interaction  and  the 
same  object  after  interaction.  (Langford  2013,  308) 

(6)  Objectives/Missions 

Bornman  (1993,  16)  describes  objectives  or  missions  as  the  descriptive  tenns  used 
to  identify  desired  end  states  or  achievements  as  a  result  of  employing  operational  forces. 

B.  C2  SYSTEM  IN  LAND  BATTLEFIELD 

1.  Network-Centric  Warfare 

It  is  important  to  understand  the  significant  contributions  of  communication 
towards  measuring  the  effectiveness  of  C2.  The  commander  communicates  with  his 
subordinate  for  task  assignment  (command)  and  the  subordinate  communicates  with  the 
commander  for  task  update  during  task  monitoring  (control)  and  this  communication 
must  be  maintained  consistently  to  ensure  the  task  is  accomplished  within  the  allocated 
resources  and  constraint.  Consider  the  combat  team  mission  in  the  land  battlefield.  The 
combat  team  consists  of  four  land  platforms  connected  with  one  another  in  the  combat 
team  network.  In  addition,  the  commander  of  the  combat  team  is  connected  to  the  higher 
command,  i.e.,  HQ  in  the  command  post,  via  the  command  network.  Through  constant 
communication  (position  update,  task  status  update,  and  combat  status  update)  between 
the  commander  and  his  subordinates  and  the  commander  to  the  higher  command,  the 
commander  is  able  to  establish  the  situation  awareness  in  the  battlefield  to  make 
informed  decision.  In  addition,  the  communication  must  be  reliable  (with  minimal  or  no 
disruption  due  to  interference  caused  by  terrain,  canopy  in  the  forested  area  and  signal 
jamming)  and  secured  (using  message  encryption  and  frequency  protection)  to  establish 
trust  between  the  commanders  and  the  subordinates  to  accomplish  task. 

As  the  result  of  technological  advancements  with  the  transition  from  stand-alone 
systems  to  increasingly  interconnected  systems,  operation  troops  nowadays  are  well 
connected  in  the  infonnation  networks  to  perform  their  tasks.  NCW  is  described  by  the 
Department  of  Defense  (DOD  2005)  as  follows: 

Network-centric  warfare  is  an  emerging  theory  of  war  in  the  Information 

Age.  It  is  also  a  concept  that,  at  the  highest  level,  constitutes  the  military’s 
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response  to  the  Information  Age.  The  term  network-centric  warfare 
broadly  describes  the  combination  of  strategies,  emerging  tactics, 
techniques,  and  procedures,  and  organizations  that  a  fully  or  even  a 
partially  networked  force  can  employ  to  create  a  decisive  warfighting 
advantage.  (DOD  2005,  3) 

2.  System  of  Systems 

In  the  network-centric  driven  environment,  there  is  an  increasing  need  for 
interoperability  between  the  C2  system  and  other  military  systems.  For  instance,  the 
commanders  in  the  armor  combat  team  can  link  up  and  collaborate  via  an  ad-hoc  tactical 
network,  gain  better  awareness  of  the  enemy  situation  using  the  payload  information 
from  the  unmanned  aerial  vehicle  (UAV),  send  information  back  to  higher  command  via 
the  satellite  link,  and  fuse  intelligences  to  establish  the  common  situational  picture. 
Therefore,  the  C2  system  has  emerged  as  a  System  of  Systems  (SoS). 

Maier  (n.d.)  describes  SoS  as: 

SoS  should  be  distinguished  from  large  but  monolithic  systems  by  the 
independence  of  their  components,  their  evolutionary  nature,  emergent 
behaviors,  and  a  geographic  extent  that  limit  the  interaction  of  their 
components  to  infonnation  exchange.  Within  these  properties  are  further 
subdivisions.  For  example,  a  distinction  between  systems  which  are 
organized  and  managed  to  express  particular  functions,  and  those  in  which 
desired  behaviors  must  emerge  through  voluntary  and  collaborative 
interaction. 

Maier  (n.d.)  further  distinguishes  SoS  from  large  and  complex  but  monolithic 
systems  with  the  following  five  characteristics: 

a.  Operational  Independence  of  the  Elements 

The  SoS  is  comprises  of  several  independent  systems,  which  are  functional  at 
stand-alone  deployment.  For  instance,  consider  the  integration  between  the  C2  system 
and  the  platform  navigation  system  in  which  the  C2  system  collects  positional  updates 
from  the  navigation  system  and  broadcast  to  other  friendly  forces  for  blue  force  tracking. 
If  the  connection  between  the  C2  system  and  the  navigation  system  is  broken  or 
intermittent  (e.g.,  loose  wiring  connection),  the  operator  would  be  able  to  share  his  or  her 
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location  through  manual  means  (compass  and  map),  and  the  navigation  system  would 
still  be  able  to  receive  a  geographical  location  from  the  satellite. 

b.  Managerial  Independence  of  the  Elements 

Consider  the  tactical  network  that  connects  the  C2  systems  in  the  mobile  land 
platforms  (e.g.,  infantry  fighting  vehicle  or  main  battle  tank).  The  network  system 
governs  the  membership  of  the  platforms  such  that  the  platforms  can  join  the  network  via 
governed  network  protocols,  message  format,  and  operation  bandwidth  (e.g.,  via 
UHF/VHF/satellite  connection)  for  task  collaboration.  On  completion  of  the 
collaboration,  the  platfonns  can  leave  the  network  to  execute  individual  tasks.  While 
these  component  systems  are  able  to  operate  as  a  SoS  when  they  are  integrated  together, 
the  subsystems  continue  to  maintain  individual,  independent  operational  existence  while 
being  a  part  of  the  SoS. 

c.  Evolutionary  Development 

Consider  the  geographical  challenges  of  implementing  cooperative  engagement 
capabilities  (CEC)  in  the  land  battlefield  context  where  lines  of  sight  (LOS)  among  the 
platforms  in  the  combat  team  are  hindered  by  terrain  and  vegetation,  thus  affecting  team 
communication  and  threat  detection,  classification,  tracking,  and  engagement.  Through 
ongoing  technological  research  and  development,  the  CEC  in  the  form  of  SoS  evolves 
gradually. 

d.  Emergent  Behavior 

Consider  the  long-range  sense-and-strike  mission  in  which  the  UAV  pipes  the 
surveillance  picture  to  the  C2  system  in  the  armor  battle  group  via  the  satellite  link.  The 
commander  assesses  the  situation  and  executes  an  artillery  attack  command  via  the 
satellite  link  or  tactical  network  to  the  artillery  battery  to  engage  the  threat.  In  this 
scenario,  the  combination  of  the  functions  leads  to  an  emergence,  i.e.,  new  sense-and- 
strike  capability,  that  cannot  be  performed  by  each  individual  system  (the  UAV  system, 
the  C2  system,  and  the  artillery  firing  system). 


10 


e. 


Geographic  Distribution 


Consider  Satcom-on-the-Move  (SOTM),  which  is  the  key  enabler  for  long 
distance  communication  in  the  land  battlefield.  Land  platforms  equipped  with  the 
ruggedized  onboard  satellite  antenna  are  able  to  communicate  C2  information 
(intelligence  on  enemy  situation  report)  over  a  long  range  back  to  the  HQ  at  high  data 
bandwidth. 

C.  STAKEHOLDER  ANALYSIS 

Consider  a  simplified  stakeholder  analysis  for  C2  system  program  where  there  are 
three  groups  of  stakeholders,  the  Anny  capability  development  office,  the  program  team, 
and  the  defense  contractor  team,  as  illustrated  in  Figure  2. 

Figure  2.  Stakeholder  Analysis  for  a  Typical  C2  System  Program 


Program 


Defense 


Engineer 


The  end-user  (the  Army)  need  for  the  C2  system  to  provide  them  with  situational 
awareness  and  decision  making  aids  is  identified  as  the  most  important  requirement  in  the 
system.  Both  Army  commander  and  Army  subordinate  use  the  C2  system  during  mission 
operations  (the  mode  of  usage  shall  be  discussed  in  a  subsequent  section).  The  program 
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manager  works  with  the  Army  on  the  development  of  the  capability  and  the  needs 
identification  for  the  C2  system.  When  the  system  concept  for  a  development  item  is 
defined  and  commented  on  through  the  acquisition  process,  the  acquisition  authority 
subsequently  contracts  with  the  defense  contractor(s)  to  further  refine,  develop,  and 
deliver  the  system.  Therefore,  with  domain  expertise  in  the  capability  and  operational 
area,  the  program  team  is  in  charge  of  architecting  the  system  at  the  capability  and 
operational  level,  and  the  system  architect  under  the  defense  contractor  team  (software 
provider,  hardware  vendor,  system  integrator)  shall  assume  the  responsibility  of 
developing  the  functional  and  physical  architecture  (interface  between  subsystems, 
software  architecture,  power  distribution,  network  connection  and  messaging  protocol). 
The  engineer  (software,  hardware,  systems)  develops  the  C2  system  based  on  the 
approved  architecture  design  by  the  stakeholders. 

The  acquisition  processes,  the  development  processes,  and  the  operational 
processes,  each  contribute  to  the  effectiveness  of  the  operational  system.  That 
effectiveness  can  be  reflected  in  two  domains — the  functional  domain  and  the  process 
domain.  The  functional  domain  reflects  the  stakeholders’  functional  performances  while 
the  process  domain  reflects  the  stakeholders’  processes.  Functional  performances  are 
measurable  and  quantifiable,  whereas  the  process  activities  are  verifiable.  Thus,  the  role 
of  the  program  manager  is  captured  through  the  functions  and  processes  of  TO 
MANAGE  (Langford  2012,  138-146).  For  each  of  the  stakeholders  a  set  of  functions  and 
processes  are  designated  as  those  having  the  most  influence  on  effectiveness.  That 
designation  of  importance  is  determined  by  the  subject  matter  experts  who  are  most 
familiar  with  the  implementation  of  the  functions  that  are  deem  most  important  to  the 
outcomes  associated  with  each  life  cycle  phase  in  a  system’s  development.  See  Figure  5 
for  a  general  depiction  of  measures  of  effectiveness  and  subsequent  discussion  for  the 
specifics. 

A  key  aspect  in  determining  effectiveness  of  a  C2  system  is  based  on  the  system 
architecture.  The  life  cycle  phases  of  a  C2  architecture  are  first  driven  by  the 
stakeholders’  needs.  Stakeholders’  perspective  should  reflect  their  individual  needs,  that 
when  combined  form  the  operating  of  a  C2  system.  For  example,  from  the  operators’ 
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perspective  (army  commander  and  subordinate),  the  desired  expectations  of  the  C2 
system  are  performance,  interoperability,  ease  of  use,  maintenance,  and  system  safety. 
The  C2  systems  shall  be  able  to  integrate  and  be  interoperable  (“plug  and  play”)  with 
other  subsystems  (e.g.,  new  weapon  system  and  communication  system)  with  minimal 
configuration  change.  Measures  of  effectiveness  (MOEs)  and  functional  measures  of 
performance  (MOPs)  are  evaluated  and  given  to  the  program  manager,  as  feedback  to 
ensure  compliance  with  stakeholder  needs  to  ensure  that  the  C2  system  performs  to  its 
desired  capability. 

From  the  program  manager’s  perspective,  providing  value-added  services  and 
meeting  operators’  expectations  are  the  driving  factors  towards  a  service-oriented 
architecture  (SOA)  that  supports  a  high  degree  of  reusability.  In  addition,  new  sensor 
systems  can  be  integrated  with  the  C2  system  with  minimal  code  revision  as  the 
processing  units  are  developed.  However,  there  is  a  need  to  ensure  modularity  during 
software  development  so  that  reuse  of  the  software  module  is  possible. 

While  providing  a  robust  C2  system  using  SOA,  there  is  also  a  need  to  ensure 
information  assurance,  especially  when  there  is  communication  between  the  commander 
and  the  subordinate  over  a  tactical  network  during  operation.  There  is  a  need  to  encrypt 
the  data  message  so  that  information  exchange  is  not  compromised.  With  message  size 
increased  for  transmission,  the  effective  information  exchange  would  inevitably  be 
affected.  In  addition,  with  the  need  for  authentication,  virus  scanning,  and  restricted 
customization,  the  ease  of  use  is  also  affected.  Furthermore,  there  is  a  need  to  protect  the 
security  policy  implementation  through  employing  selected  defense  contractors,  and  a 
need  for  new  subsystem  integration.  There  is  also  a  need  to  conduct  extensive  tests  to 
ensure  the  security  integrity  of  the  system  is  not  compromised,  thus  lengthening  the 
development  life  cycle.  Hence,  the  advantage  of  adopting  SOA  to  shorten  development 
time  is  traded-off  with  longer  testing  for  information  assurance.  The  implications  of 
Figure  2  to  the  overall  considerations  for  measuring  effectiveness  are  apparent  in  the 
interactions  between  the  stakeholders. 
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D.  SAFETY  STANDARD 

As  the  C2  system  is  most  often  dominated  by  software  development,  there  is  a 
need  to  ensure  that  the  C2  software  architecture  conforms  to  system  safety  practices,  such 
as  MIL-STD-882E  (DOD  2012).  The  practice  facilitates  stakeholder  communication  in 
term  of  hazard  identification  and  analysis  so  that  proper  measures  are  taken  to  pre-empt 
or  mitigate  the  hazard. 

Software  is  defined  in  MIL-STD-882E  (DOD  2012)  as 

a  combination  of  associated  computer  instructions  and  computer  data  that 
enable  a  computer  to  perfonn  computational  or  control  functions. 
Software  includes  computer  programs,  procedures,  rules,  and  any 
associated  documentation  pertaining  to  the  operation  of  a  computer 
system.  Software  includes  new  development,  complex  programmable 
logic  devices  (firmware),  Non-Developmental  Item  (NDI),  Commercial- 
off-the-Shelf  (COTS),  Government-off-the-Shelf  (GOTS),  re-used, 
Government-Furnished  Equipment  (GFE),  and  Government-developed 
software  used  in  the  system. 

Tables  1  and  2  represent  the  software  safety  critical  matrix  that  establishes  the 
Software  Criticality  Indices  (SwCIs).  The  score  of  the  indices  determines  the  expected 
level  of  rigor  (LOR)  tasks  to  be  carried  out  to  address  the  safety  related  issue.  Each  SwCI 
recommends  the  level  of  rigor  task  to  be  performed  to  ensure  the  C2  system  is  safe  for 
operation. 


Table  1.  Software  Safety  Criticality  Matrix  (from  DOD  2012,  16) 


SOFTWARE  SAFETY  CRITICALITY  MATRIX  (from  DOD  2012,  16) 

SEVERITY  CATEGORY  (from  DOD  2012,  16) 

Software 

Control 

Category 

Catastrophic  (1) 

Critical  (2) 

Marginal  (3) 

Negligible  (4) 

1 

SwCI  1 

SwCI  1 

SwCI  3 

SwCI  4 

2 

SwCI  1 

SwCI  2 

SwCI  3 

SwCI  4 

3 

SwCI  2 

SwCI  3 

SwCI  4 

SwCI  4 

4 

SwCI  3 

SwCI  4 

SwCI  4 

SwCI  4 

5 

SwCI  5 

SwCI  5 

SwCI  5 

SwCI  5 
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The  software  control  category  (score  1  to  5)  is  determined  by  the  degree  of 
automation  that  the  software  function  provides  to  the  operational  command.  Consider  the 
firing  command.  The  score  of  1  implies  that  the  software  has  full  control  over  the 
function,  i.e.,  the  firing  command  is  automated  by  the  software,  and  the  score  of  5  implies 
the  software  function  does  not  have  any  influence  or  impact  to  the  firing  command.  This 
score  is  compared  to  the  severity  category  to  determine  the  LOR  required  to  ensure  the 
potential  hazard  is  either  removed  or  mitigated  through  procedures.  For  instance,  if  the 
software  function  has  influence  on  the  weapon  system,  i.e.,  the  operator  can  issue  a  firing 
command  using  the  C2  software  function,  this  function  would  be  allocated  a  score  of  2  or 
3  through  stakeholders’  consensus.  The  potential  hazard  identified  from  this  function  is 
occurrence  of  firing  on  friendly  force  in  the  event  when  the  operator  perceives  the 
friendly  force  as  threat  in  the  C2  situational  picture  and  issues  a  firing  command.  The 
severity  of  this  hazard  is  considered  Catastrophic  (1).  Therefore,  the  SwCI  is  considered 
as  1  and  the  level  of  rigor  to  be  conducted  on  this  function  to  ensure  system  safety  is 
displayed  in  Table  2: 


Table  2.  Level  of  Rigor  Tasks  (from  DOD  2012,  16) 


SwCI 

Level  of  Rigor  Tasks 

SwCI  1 

Program  shall  perfonn  analysis  of  requirements,  architecture, 
design,  and  code;  and  conduct  in-depth  safety-specific  testing. 

SwCI  2 

Program  shall  perform  analysis  of  requirements,  architecture,  and 
design;  and  conduct  in-depth  safety-specific  testing. 

SwCI  3 

Program  shall  perfonn  analysis  of  requirements  and  architecture; 
and  conduct  in-depth  safety-specific  testing. 

SwCI  4 

Program  shall  conduct  safety-specific  testing. 

SwCI  5 

Once  assessed  by  safety  engineering  as  Not  Safety,  then  no  safety- 
specific  analysis  or  verification  is  required. 

E.  LIMITATION  AND  CONSTRAINT 
1.  Communication 

Communication  is  the  key  element  for  two  or  more  stations  to  exchange 


information.  Typical  information  is  exchanged  in  two  manners:  either  through  voice 
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communication  or  data  communication  in  the  form  of  message  in  byte  or  bit.  The 
information  exchanged  can  be  the  current  location,  target  position,  or  request  for 
replenishment  support,  for  example.  Consider  the  different  kinds  of  communication:  a 
fleet  commander  requesting  the  location  of  his  fleet  for  better  situational  awareness;  a  fire 
support  officer  requesting  an  air  strike;  a  logistics  officer  requesting  replenishment;  an 
intelligence  officer  requesting  the  latest  update;  or  a  commanding  officer  requesting  the 
status  update  to  support  his  troops.  The  data  message  can  be  exchanged  via  various 
means,  such  as  tactical  combat  radio,  re-broadcast  station,  and  satellite  connection. 

The  C2  system  is  generally  a  collaborative  system  between  the  commander  and 
his  subordinates.  The  following  heuristics  that  the  collaborative  system  should  possess,  as 
described  by  Maier  (Maier  1999),  are  used  to  critique  the  communications  portion  of  the 
C2  system  architecture: 

Ensuring  cooperation:  A  collaborative  system  exists  because  the  partially 
independent  elements  decide  to  collaborate.  The  designer  must  consider 
why  they  will  choose  to  collaborate  and  foster  those  reasons  in  the  design 
(Maier  and  Rechtin  2009). 

2.  Stability 

Consider  the  mission  during  the  battle.  There  is  often  a  need  for  the  commander 
to  maintain  constant  communication  with  his  troops  to  ensure  timely  commands  are  given 
and  control  is  maintained  consistently.  Interruption  occurs  whenever  there  is  a  break  in 
communication,  such  as  blockage,  terrain  masking,  and  equipment  breakdown. 
Maintaining  stability  in  C2  is  fundamental  to  task  completion. 

Langford  (2012,  111)  describes  stability  in  a  system  as: 

For  a  system  to  exist  and  sustain  itself  as  a  system,  it  requires  some 
semblance  of  metastability,  i.e.,  dynamic  stability  to  continue  as  system. . . 
Metastability  is  the  intrinsic  property  of  a  group  of  objects  that  persists  in 
an  apparent  equilibrium  of  interactions  between  objects  where  only  a 
small  disturbance  in  the  established  interaction  can  dramatically  change 
(reduce  or  increase)  the  system’s  lifetime.  (Langford  2012,  111) 

System  metastability  is  important  to  ensure  the  success  of  the  mission.  During  the 
collaboration,  information  would  not  be  exchanged  effectively  when  there  is  constant 
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interruption  caused  by  network  failure,  equipment  failure,  or  software  failure.  The 
mission  is  deemed  a  failure  because  the  system  cannot  sustain  itself  throughout  the 
lifetime.  The  heuristic  shared  by  Maier  and  Rechtin  (Maier  and  Rechtin  2009),  which  the 
collaborative  system  should  possess,  is  used  as  the  critique  for  the  C2  system 
architecture.  It  reads: 

Stable  intermediate  forms:  A  collaborative  system  designer  must  pay 
closer  attention  to  the  intermediate  steps  in  a  planned  evolution.  The 
collaborative  system  will  take  on  intermediate  forms  dynamically  and 
without  direction  as  part  of  its  nature.  (Maier  and  Rechtin  2009) 

Given  the  land  battlefield  environment  where  communication  is  limited  by  the 
terrain,  canopy,  and  vegetation,  it  is  common  to  lose  connection  between  the  commander 
and  the  subordinate.  As  a  result,  cooperation  is  limited  due  to  a  physical  communication 
breakdown.  All  stakeholders  indicated  in  Figure  2  agree  there  is  a  need  to  minimize  loss 
of  connection  by  typically  employing  additional  tactical  radio  with  different  frequency 
wavelengths  for  redundancy,  deploying  a  re-broadcast  station  to  bridge  the 
communication,  or  deploying  more  forces  to  relay  communication. 

However,  each  action  incurs  additional  manpower  and  resource  cost  in  which  the 
commander  must  consider  and  prepare  to  tradeoff  for  continual  communication.  Thus, 
there  is  a  need  to  evaluate  the  MOE  of  using  the  C2  system  in  light  of  the  increasing 
requirement  for  more  subsystem  integration. 

This  chapter  has  provided  the  background  studies  of  the  C2  system  and  the 
challenges  to  meet  user  expectation.  The  next  chapter  shall  emphasis  on  the  literature 
review  of  MOEs. 
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III.  LITERATURE  REVIEW  MEASURES  OF  EFFECTIVENESS 


A.  MEASURES  OF  EFFECTIVENESS  AS  TECHNICAL  MEASUREMENTS 

Roedler  and  Jones  (2005,  9)  define  measures  of  effectiveness  (MOEs)  as 
“operational  measures  of  success  that  are  closely  related  to  the  achievement  of  mission  or 
operational  objectives.”  They  refer  MOEs  to  “a  subset  of  technical  measurement,”  as 
illustrated  in  Figure  3,  which  is  the  set  of  measurement  activities  used  to  provide  the 
stakeholder  with  insight  into  the  progress  of  defining,  developing,  and  sustaining  the 
product  or  service.  In  a  general  sense,  measures  of  effectiveness  are  “intended  to 
determine  to  what  extent  objectives  are  accomplished  and  how  well  the  results  compare 
with  the  desired  results”  (Roedler  and  Jones  2005,  9). 

Roedler  and  Jones  (2005,  6)  include  the  following  as  technical  measurement: 

•  KPPs  are  a  critical  subset  of  the  performance  parameters,  representing  the  most 
critical  capabilities  and  characteristics. 

•  MOPs  characterize  the  physical  or  functional  attributes  related  to  the  system 
operation. 

•  TPMs  measure  attributes  of  a  system  element  within  the  system  to  determine  how 
well  the  system  or  system  element  is  satisfying  specified  requirements. 
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Figure  3.  Technical  Measures  Relationships  (from  Roedler  and  Jones  2005,  15) 


Technical 

Insight 

(Progress 

&Risk) 


MOEs  are  used  to: 

•  Compare  operational  alternatives. 

•  Investigate  performance  sensitivities  to  changes  in  assumptions  from  the 
user’s  view. 

•  Define  operational  performance  requirements. 

•  Evaluate  the  achievement  of  key  operational  performances. 

•  Serve  as  the  standard  of  acceptance  and  evaluation  for  the  technical 
solution.  (Roedler  and  Jones  2005,  9-10) 

Roedler  and  Jones  (2005)  provide  a  set  of  guidelines  to  select  MOEs,  as  listed  in 
Table  3. 
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Table  3.  Measures  of  Effectiveness  Selection  Guidelines  (from  Roedler  and 

Jones  2005, 36) 

_ MOE  Selection  Guidelines _ 

Provides  insight  into  at  least  one  operational  objective  or  mission  requirement, _ 

MOEs  should  not  be  strongly  correlated;  they  should  provide  insight  into  different 
aspects  of  the  operational  alternative. _ 

Select  and  define  in  the  context  of  the  operational  objective:  no  predefined  MOEs/ 
values. _ 

Select  and  define  independent  of  the  alternatives  at  hand;  represent  an  independent  means 
to  collectively  evaluate  the  alternatives. _ 

Select  only  a  few  MOE/MOPs,  which  may  be  an  order  of  magnitude  more  TPMs. _ 

Each  KPP  should  have  an  associated  MOE  or  MOP. 


B.  MEASURES  OF  EFFECTIVENESS  OPERATIONAL  TESTING  AND 

EVALUATION 

Operational  testing  and  evaluation  (OT&E)  is  considered  as  a  major  milestone  in 
the  project  implementation  cycle.  The  success  of  project  implementation  relies  heavily  on 
the  completion  of  the  OT&E  because  OT&E  usually  takes  up  a  large  portion  of  the 
payment  milestone  from  contractor’s  perspective.  From  the  user’s  perspective,  the  OT&E 
completion  provides  the  level  of  confidence  on  the  system  readiness  to  be  deployed  in  the 
field.  Therefore,  it  is  essential  that  the  system  MOE  be  addressed  as  part  of  OT&E 
completion. 

In  the  OT&E  context,  Stevens  (1986)  defines  MOEs  as  “any  set  of  criteria 
established  to  determine  the  resolution  of  a  critical  issue.”  Generally,  the  OT&E  passing 
criteria  set  the  guidelines  to  develop  the  MOEs.  Stevens  (1986)  lists  the  following 
characteristics  to  describe  a  good  MOE  and  seven  MOE  development  rules  as  tabulated 
in  Tables  4  and  5,  respectively. 
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Table  4. 


Characteristics  of  a  Good  Measure  of  Effectiveness  (from  Stevens 

1986,55) 

_ Characteristics  of  a  Good  MOE _ 

The  MOE  should  be  relevant  (Stevens  1986,  55). _ 

The  set  of  MOE  should  be  complete  (Stevens  1986,  55). _ 

The  MOE  should  be  precisely  defined  (Stevens  1986,  55). _ 

The  MOE  sets  should  be  mutually  exclusive  (Stevens  1986,  55). _ 

The  MOE  should  be  expressed  in  terms  that  are  meaningful  to  testers  and  developers 
(Stevens  1986,  55). _ 

MOE  meaning  should  not  be  open  to  interpretation  with  the  passage  of  time  (Stevens 
1986,  55). _ 

MOE  inputs  should  be  measureable  (Stevens  1986,  55). _ 


Table  5.  Measure  of  Effectiveness  Development  Rules  (from  Stevens  1986, 

55-56) 

_ MOE  Development  Rules _ 

There  should  be  one  MOE  for  each  mission  capability. _ 

Decision-makers  shall  assign  MOE  weights. _ 

Missions/scenarios  should  be  fully  defined  before  measurement  collection  during  testing. 

Measurements  should  not  interfere  with  system  operation. _ 

MOE  quantitative  measurements  should  be  stated  as  probabilities. _ 

All  qualitative  measurements  should  use  the  same  standard. _ 

When  recording  system  failures  during  testing,  include  system  &  hardware  failures. 


C.  MEASURES  OF  EFFECTIVENESS  AS  THE  FULFILLMENT  OF 

STAKEHOLDERS’  NEEDS 

Sproles  (2002)  proposed  the  MOE  development  should  “evaluate  the  performance 
of  the  system  in  relation  to  mission  accomplishment,”  with  focus  and  emphasis  on 
stakeholder  needs  and  requirements.  Sproles  (2002)  emphasized  the  use  of  MOEs  to 
address  critical  operational  issues  (COIs),  defined  “as  an  emergent  property  that  the 
system  must  have  in  order  to  perfonn  its  function  [and]  that  a  solution  to  a  need  must 
possess  in  order  to  meet  the  need”  (256-57).  Sproles  further  clarified  the  difference 
between  the  formulation  of  the  MOE  and  MOP: 


22 


An  MOE  refers  to  the  effectiveness  of  a  solution  and  is  independent  of  any 
particular  solution;  an  MOP  refers  to  the  actual  performance  of  an  entity. . . 

An  MOE  will  indicate  a  property  which  a  potential  solution  must  possess 
in  order  to  meet  a  need:  An  MOP  will  tell  what  something  is  capable  of 
doing,  even  if  this  is  not  necessarily  what  the  stakeholders  want  it  to  do. 

The  difference  between  effectiveness  and  performance  as  applied  to  a 
solution  to  a  need  is  that  effectiveness  is  a  quality  of  fitness  for  service  or 
of  producing  the  results  for  which  it  was  intended.  Perfonnance  is  the 
quality  of  doing  something,  and  doing  something  does  not  necessarily 
indicate  fitness  for  service.  (Sproles  2000,  56-57) 

The  MOE  development  process  as  proposed  by  Sproles  is  the  system  engineering 
approach  that  begins  with  a  series  of  reviews  from  gathering  users’  requirements,  design, 
and  feasibility  studies  to  determine  the  most  effective  solution.  Once  the  solution  is 
selected,  the  MOEs  are  then  developed  to  evaluate  the  effectiveness  of  the  system  to 
operate  during  the  mission.  Figure  4  illustrates  the  development  process  of  the  MOEs 
from  Sproles. 


Figure  4.  Measure  of  Effectiveness  Development  Process  (from  Sproles  2002, 

255) 
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However,  an  MOE  developed  through  this  process  is  geared  towards  the  COIs, 
which  are  driven  by  the  stakeholders’  perspective  of  the  system;  therefore,  the  application 
of  the  MOE  is  limited  solely  to  mission  performance. 

D.  MEASURES  OF  EFFECTIVENESS  AS  DETERMINANTS  OF  THE 

INTEGRATIVE  FRAMEWORK 

MOE  is  defined  as  “the  single-most  often  touted  and  applied  method  of  thought  of 
how  well  one  is  doing”  (Langford  2014,  8).  Langford  explained  that  “the  what”  to 
measure  for  effectiveness  must  be  interpreted  within  a  particular  context,  regardless  of 
type.  Without  boundaries,  it  is  uncertain  what  dimensionality  means  in  terms  of 
effectiveness,  let  alone  how  it  could  be  compared  and  how  meaningful  such  a  comparison 
might  be.  And  “the  how”  and  “the  when”  are  seemingly  attached  to  milestones  in  the 
acquisition  parlance  rather  than  to  the  merits  of  any  theoretical  foundation.  Langford 
(2014)  proposed  developing  MOEs  using  an  integrative  framework  as  illustrated  in 
Figure  5  (Langford  2014,  8-12).  Within  the  integrative  framework  (objects  and 
processes),  each  of  the  intersections  between  objects  and  processes  has  a  set  of  MOEs 
(Langford  2014,  8-12).  The  first  set  is  related  to  the  process  frame.  MOEs  that  are  related 
to  processes  are  verifiable  and,  as  such,  are  not  quantifiable.  The  second  set  of  MOEs  is 
related  to  the  object  frame.  MOEs  that  are  related  to  objects  (e.g.,  physical  objects, 
functions,  and  behaviors)  are  quantifiable.  One  insight  observed  from  the  integrative 
framework  of  MOEs  is  normally  seen  as  being  quantifiable,  but  in  fact,  half  of  them  are 
not.  For  example,  the  measure  of  effectiveness  for  physical  objects  and  functions  is 
quantifiable  through  data  measurement,  i.e.,  mission  log  collection,  and  database,  and  the 
analysis  of  the  data  provides  the  objective  value  in  detennine  whether  the  objects  or 
functions  have  fulfilled  or  exceed  the  user  expectation  during  task  execution.  As  for 
behaviors,  i.e.,  leadership  in  command  and  control,  style  in  task  execution,  it  is  rather 
towards  the  subjective  approach  to  evaluate  the  effectiveness  in  mission  accomplishment. 
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Figure  5.  Integrative  Framework  (from  Langford  2014,  7) 
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As  illustrated  in  Figure  6,  fitness-for-purpose  is  defined  as  “the  real  knowledge 
that  a  potential  user  needs  to  have  is  whether  the  product  or  service,  decision  or 
judgment,  plan  or  outcome,  technology  or  engineering  is  good  for  a  purpose”  (Langford 
2014,  18). 


Figure  6.  Linkage  between  Measures  of  Effectiveness  and  Fitness-for-Purpose 

(from  Langford  2014,  8) 
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This  chapter  has  provided  the  literature  review  of  various  MOE  development 
processes  and  techniques.  Langford’s  integrative  framework  shall  be  used  in  the  next 
chapter  for  MOE  development. 
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IV.  METHODOLOGY  TO  DEVELOP  MEASURE  OF 

EFFECTIVENESS 


The  previous  chapter  provides  the  literature  review  of  various  MOE  development 
processes  and  techniques.  With  technological  advances  in  the  sensor  and  processing 
systems,  there  is  a  growing  interest  to  integrate  these  systems  into  the  C2  system  to 
achieve  better  and  faster  situational  awareness  and  decision  making  aid.  However,  there 
is  a  need  to  evaluate  the  outcomes  (performance  and  experience)  to  determine  the  value 
(subjective  and  objective)  of  fulfilling  the  requirements.  These  evaluations  would  better 
manage  the  user  expectations  in  tenns  of  the  return  on  investment  to  perform  the 
integration.  Thus,  MOEs  are  widely  applied  to  measure  and  evaluate  the  performance  of 
C2  systems  in  a  combat  context.  This  chapter  covers  the  nine-step  methodology  using 
integrative  framework  to  develop  MOE  for  evaluating  the  effectiveness  of  the  C2  system 
in  the  land  battlefield. 

A.  NINE-STEP  METHODOLOGY 

The  measures  of  effectiveness  (MOEs)  of  C2  systems  on  land  battlefields  are 
developed  and  evaluated  using  the  nine-step  methodology.  The  nine-step  methodology 
was  developed  to  unify  the  concepts  of  MOEs  into  a  repeatable,  validated  process  and  to 
identify  MOEs  associated  with  both  development  and  operations  (Langford  2014,  10).  A 
flow  diagram  of  the  nine-step  methodology  is  displayed  in  Figure  7.  Using  the  nine-step 
methodology  and  the  integrative  framework,  12  pairs  of  MOEs  are  developed. 
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Figure  7.  Nine-Step  Methodology  to  Characterize  MOEs  (from  Langford  2014, 

14) 
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The  nine  steps  are  described  below  (Langford  2014,  10-13): 

(1)  Define  Terminology 

These  terms  that  are  used  in  Ligure  5  are  mathematically  defined  and  formalized 
through  predicate  calculus  (Langford  2014).  The  result  is  a  consistent  set  of  defined  terms 
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that  combine  to  provide  a  validity  structure  for  the  integrative  framework  and  the  MOEs 
and  MOPS. 


(2)  Delineate  Boundaries  and  Functions 

The  functional  decomposition  of  “To  Command”  and  “To  Control”  are  performed 
and  analyzed.  The  boundaries  are  described  through  physical,  functional,  and  behavioral 
domains  (Langford  2012).  The  problem  domain  lies  within  these  three  types  of 
boundaries.  For  the  C2  system,  the  boundaries  extend  beyond  the  developed  system  to 
the  domain(s)  of  the  adversary.  The  MOEs  extend  to  the  boundaries  and  encapsulate  all 
functions  and  solutions  (Langford  2012). 

(3)  Perform  Life  Cycle  Analysis 

In  the  language  of  life  cycle  analysis,  functions  are  measured  and  quantified  from 
the  different  perspectives  of  the  stakeholders.  The  function  of  ‘to  receive’  is  different 
from  the  function  ‘to  transmit’,  but  both  functions  are  necessary  for  the  higher  level 
function  of ‘to  communicate’.  Life  cycle  analysis  is  best  performed  at  the  highest  level  of 
abstraction  within  the  functional  domain  so  as  to  capture  the  overall  perspective  of  the 
MOEs. 


(4)  Define  Requirements 

This  section  specifies  the  requirements  that  satisfy  stakeholder  needs  (Langford 
2014).  These  requirements  would  be  used  for  MOE  traceability  and  validation.  There  are 
commensurate  MOEs  for  that  capture  processes  that  must  occur.  These  processes  are 
verifiable — either  performed  or  not  performed  (Langford  2014). 

(5)  Postulate  Solution  Set 

This  section  conceptualizes  and  characterizes  a  set  of  solutions  that  satisfy 
stakeholder  needs  (Langford  2014).  The  set  of  alternative  solutions  exists  within  the 
boundaries  of  the  problem. 
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(6)  Determine  Theoretical  Foundation 

MOEs  are  sensitive  to  the  theory  on  which  measurements  are  referenced. 
Logically,  the  theoretical  underpinning  should  be  consistent  throughout  the  boundaries, 
so  that  the  MOEs  have  structure,  semantic,  and  syntax  validity  (Langford  2014). 

(7)  Formalize  Integrative  Framework 

This  section  maps  the  processes  (used  in  the  MOEs)  to  physical  objects  (functions 
used  in  MOEs)  through  the  integrative  framework  (Langford  2014).  Langford  (2014) 
describes  the  framework  as  follows: 

The  framework  captures  the  product  and  management  needs  in  the  nexus 
formed  by  the  intersection  of  an  objective  frame  and  a  subjective  frame. 

Within  the  integrative  framework  (objects  and  processes),  each  of  the 
intersections  between  objects  and  process  has  a  pair  of  MOEs.  The  first  set 
is  related  to  the  process  frame.  MOEs  that  are  related  to  processes  are 
verifiable,  and  as  such  are  not  quantifiable.  The  second  set  of  MOEs  is 
related  to  the  object  frame.  MOEs  that  are  related  to  objects  (e.g.,  physical 
objects,  functions,  and  behaviors)  are  quantifiable.  We  normally  think  of 
MOEs  as  being  quantifiable,  but  in  fact  half  of  them  the  processes)  are  not 
quantifiable  (only  verifiable).  Figure  8  displays  the  framework’s  two 
frames:  the  object  (objective)  frame  and  the  process  (subjective)  frame. 

The  arrows  in  Figure  8  indicate  the  interaction  sequencing  as  the 
framework  is  applied  when  developing  MOEs  for  the  product  or  service. 

The  sequence  of  moving  from  the  subjective  frame  through  the  objective 
frame  illustrates  that  the  functional  analysis  portion  follows  from  the 
cognitive  processes,  the  mechanistic  process,  and  the  model  process.  The 
sequence  begins  with  cognitive  structures,  completing  a  cardinal  point 
before  moving  on  to  the  next. 
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Figure  8.  Integrative  Framework  (from  Langford  2014,  7) 


Objective 

Product 

Behaviors 

or 

Functions 

Frame 

Sendee 

Domain 

Objects 

Lz!_ 

c 


Subjective  Frame 


Social  and  Management 

Cognition 

Procedures 

-►  ^ 

Models 

7^ 

ff“ 

I 

+ 

v 


MOE 


4-M 


MOE 


MOE 


MOE 


- —  «- 


MOE 


MOE 


-+'l 


MOE 


} 


MOE 


MOE 


Each  process-object  intersection  creates  the  area  of  interest  for  the  MOE 
development  as  illustrated  in  Figure  9.  Figure  10  lists  the  MOEs  under  each  intersection 
as  described  by  Langford  (2014,  15). 


Figure  9.  Integrative  Framework  with  Measure  of  Effectiveness  Domain 
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Figure  10.  Measure  of  Effectiveness  Framework  Cardinal  Points  (from  Langford 

2014, 15) 
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B.  SUMMARY 

This  section  evaluates  the  effectiveness  of  using  a  C2  system  on  the  battlefield  to 
achieve  mission  objectives  through  the  use  of  the  MOE. 

This  chapter  walks  through  the  Langford  nine-step  methodology  for  MOE 
development.  The  next  chapter  shall  use  the  methodology  to  develop  the  MOE  for  using 
a  deployed  C2  system  in  the  land  battlefield. 
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V.  DEVELOPMENT  OF  THE  MEASURE  OF  EFFECTIVENESS 


This  chapter  discusses  the  C2  MOEs  development  using  the  Langford’s  nine-step 
methodology;  these  C2  MOEs  are  applied  to  evaluate  the  effectiveness  of  an  actual  C2 
system. 

A.  TERMINOLOGY  DEFINITION  OF  A  COMMAND  AND  CONTROL 
SYSTEM 

The  following  are  used  as  the  tenninology  definition  of  a  C2  system  to  develop 
the  MOEs: 

•  Command :  Command  is  the  art  of  assigning  missions,  prioritizing  resources,  guiding 
and  directing  subordinates,  and  focusing  the  entire  division’s  energy  to  accomplish 
clear  objectives  (Bomman  1993,  15). 

•  Control:  Control  is  the  science  of  defining  limits,  computing  requirements,  allocating 
resources,  prescribing  requirements  for  reports,  monitoring  performance,  identifying 
and  correcting  deviations  from  guidance,  and  directing  subordinate  actions  to 
accomplish  the  commander’s  intent  (Bornman  1993,  15). 

•  Command  and  Control  System :  A  command  and  control  system  consists  of  a 
collection  of  denotable  things  that  are  used  to  perform  the  C2  function:  physical 
elements  e.g.,  equipment,  such  as  transmitters  and  receivers,  computers,  a  signal 
book,  status  boards  and  other  decision  support  hardware,  code  breaking  facilities,  and 
signal  flags);  human  elements  (e.g.,  communicators,  staffs,  intelligence  analysts,  and 
the  commander  himself);  and  procedural  elements  (e.g.,  table  of  organization  and 
command  relationships,  the  content  of  a  signal  book,  and  the  content  of  a  manual  of 
training  or  doctrine)  (Hughes  1989). 

•  Measures  of  Effectiveness :  Langford  describes  MOEs  as  a  combination  of  measures 
that  embody  the  approaches  of  outcome-based,  information-based,  and  scenario- 
based  determinants.  (Langford  2014,  7). 

B.  COMMAND  AND  CONTROL  SYSTEM  BOUNDARIES  AND 
FUNCTIONS  DELINEATION 

Figures  11,  12,  and  13  illustrate  the  functional  decomposition  of  “To  Manage,” 
“To  Command”  and  “To  Control.”  Langford  (2012)  describes  command  and  control 
within  the  “To  Manage”  function. 
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Figure  1 1 .  Functional  Decomposition  of  “To  Manage” 


Functional  Decomposition  of  "To  Manage" 


Figure  12.  Functional  Decomposition  of  “To  Command” 


Functional  Decomposition  of  "To  Command" 
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Figure  13.  Function  Decomposition  of  “To  Control” 


Functional  Decomposition  of  "To  Control" 


1.  To  Command 

The  function  “to  command”  includes  the  available  C2  system  activities  required 
for  the  commander  to  lead  his  subordinates. 

a.  To  Perform  the  Act  of  Assigning  Mission 

The  commander  assigns  the  task  to  his  subordinate  using  the  C2  system. 

b.  To  Provide  Resources  (Analyze  and  Prioritize) 

The  commander  provides  the  necessary  resources  (e.g.,  logistics  support,  enemy 
situation  report,  task  priority,  and  task  difficulties)  to  the  subordinate  to  accomplish  the 
task.  Normally,  the  task  is  embedded  in  the  task  order  sent  over  the  tactical  network  using 
the  C2  system. 


c.  To  Direct  Subordinates 

The  instructions  (e.g.,  time  to  arrive,  ammunition  information,  potential  threat 
information,  task  objectives,  destination,  contingency  plan,  and  recovery  plan)  to  the 
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subordinate  to  accomplish  the  task  is  embedded  in  the  task  order  disseminated  using  the 
C2  system. 

d.  To  Analyze  Risk  (Identify  and  Assess) 

During  task  execution,  the  commander  monitors  the  progress  of  the  subordinate 
via  the  C2  system  to  identity  and  assess  any  potential  risk  that  might  jeopardize  the  task. 
The  subordinate  provides  a  periodic  task  update  to  the  commander  via  the  C2  system. 

2.  To  Control 

The  function  “to  control”  includes  the  actions  taken  by  the  commander  to  ensure 
that  the  task  assigned  to  the  subordinates  is  accomplished.  Consider  the  scenario  in  which 
the  subordinate  has  failed  to  accomplish  the  task.  The  reasons  for  task  failure  could  be 
obstacles  encountered  during  maneuver,  critical  equipment  failure,  manpower  loss,  and 
changes  in  mission  priority  from  higher  command.  In  this  situation,  the  commander  re¬ 
plans  the  task  and  assigns  the  task  to  the  subordinate  via  the  C2  system.  The  subordinate 
receives  the  task  reassignment  order  via  the  C2  system  and  proceeds  to  execute  the  task. 

a.  To  Define  Limits 

During  the  task  assignment,  the  commander  sets  the  operation  limit  (e.g., 
operating  range,  sector  of  interest,  task  duration,  and  engagement  protocol)  to  ensure  that 
subordinates  are  well  oriented  to  executing  the  task. 

b.  To  Negotiate 

The  commander  handles  any  conflicts  of  task  (e.g.,  subordinate  A  enters  into 
subordinate  B’s  task  sector)  during  task  execution.  The  C2  system  supports  the  function 
through  providing  the  common  situational  picture  to  the  commander. 

c.  To  Deal  with  Constraints 

During  a  situation  in  which  subordinates  are  unable  to  complete  the  task  due  to 
constraints,  the  commander  analyzes  the  situation,  analyzes  the  alternatives,  and 
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reassigns  the  task  to  the  subordinate  using  the  C2  system  so  that  the  mission  objective 
can  be  met. 

d.  To  Determine  Requirements 

The  commander  ensures  that  his  subordinates  are  aware  of  the  task  completion 
criteria  through  a  set  of  measurements  provided  by  the  C2  system. 

e.  To  Allocate  Resources 

During  a  situation  in  which  the  subordinates  are  unable  to  complete  the  task  due 
to  constraints,  the  commander  analyzes  the  situation  and  allocates  resources  to  the 
subordinate  via  the  C2  system  so  that  the  mission  objective  can  be  met. 

f.  To  Report 

During  task  execution,  the  subordinate  provides  a  periodic  update  to  the 
commander  on  the  task  status  via  the  C2  system. 

g.  To  Track  Performance 

The  commander  ensures  that  his  subordinates  are  aware  of  the  task  completion 
criteria  and  their  performance  according  to  the  task  instruction. 

C.  LIFE-CYCLE  ANALYSIS  OF  USING  A  DEPLOYED  COMMAND  AND 

CONTROL  SYSTEM  IN  THE  BATTLEFIELD 

Consider  the  task  assignment  scenarios  illustrated  in  Figure  14.  The  C2  process 
cycle  consists  of  four  stages:  planning,  command,  control,  and  recover.  The  commander 
plans  the  task  and  then  assigns  it  to  his  subordinates.  The  task  assignment  message  is 
transmitted  over  the  tactical  network  to  be  received  by  the  subordinate’s  station.  The 
subordinate’s  C2  system  processes  the  task  to  be  displayed  for  the  subordinate’s 
acknowledgement.  After  the  task  acknowledgement,  the  subordinates  proceed  to  execute 
the  task.  During  the  execution,  the  commander  monitors  the  progress  by  tracking  their 
movement  and  task  update.  Upon  completion  of  the  task,  the  commander  issues  a 
recovery  command  for  the  subordinates  to  recover  from  the  task.  In  the  event  when  the 
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task  is  not  executed  or  failed  to  complete,  the  commander  would  re -plan  and  reassign 
task  to  the  subordinates. 


Figure  14.  Task  Assignment  Scenarios 


Task 


D.  COMMAND  AND  CONTROL  SYSTEM  IN  THE  LAND  BATTLEFIELD 

REQUIREMENT  DEFINITION 

For  the  commander  on  the  land  platform  to  send  task  assignment  using  the  C2 
system  to  subordinates  on  another  land  platfonn,  each  land  platform  shall  be  equipped 
with  the  C2  system  connected  to  the  communication  system  for  infonnation  exchange. 
The  C2  system  shall  be  integrated  with  the  tactical  network  system  that  connects  the 
combat  team.  Each  C2  system  joins  or  leaves  the  tactical  network  using  the  operator  call- 
sign  as  identification.  During  task  execution,  the  commander  shall  be  able  to  track  the 
movement  of  the  subordinate  for  task  monitoring.  Therefore,  the  C2  system  shall  be 
connected  to  the  onboard  navigation  system  to  receive  position  information  and  broadcast 
at  regular  intervals.  The  C2  system  shall  plot  the  position  information  received  on  the 
graphical  map  in  the  display  system  to  establish  the  common  situational  picture  in  the 
combat  team.  The  C2  system  shall  be  integrated  with  the  database  system  installed  in  the 
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mission  computer  to  log  the  transaction  record  between  commander  and  subordinates 
throughout  the  mission.  The  C2  system  shall  be  integrated  with  the  onboard  sighting 
system  to  acquire  threat  range,  and  the  information  shall  be  plotted  and  broadcasted  to 
establish  common  situational  picture  in  the  combat  team.  The  C2  system  shall  allow 
operators  to  create,  edit,  view,  send,  and  acknowledge  tasks.  The  C2  system  design  shall 
conform  to  the  open  architecture  standard  to  facilitate  integration  and  collaboration  with 
other  systems.  The  C2  requirement  is  tabulated  in  Table  6. 
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Table  6.  Command  and  Control  System  Requirements 


Capabilities 

Requirements 

Description 

REQ.l 

The  system  shall  be  integrated  with  the 
communication  system  to  exchange  information 
between  units  during  a  mission. 

REQ.2 

The  system  shall  be  integrated  with  the  tactical 
network  system  to  establish  an  information 
exchange  session  for  the  combat  team  during  the 
mission.  Each  system  enters  or  leaves  the  network 
through  the  operator’s  call-sign  as  identification. 

Command  and 

REQ.3 

The  system  shall  be  integrated  with  the  navigation 
system  to  receive  position  updates  for  navigation 
during  the  mission. 

Control 

REQ.4 

The  system  shall  be  integrated  with  the  database 
system  to  store  mission  information,  such  as 
friendly  forces  track,  target  information,  and  task 
allocation. 

REQ.6 

The  system  shall  provide  a  computer  display  for 
operators  to  access  and  input  information  for  task 
assignment  and  monitoring  during  the  mission 

REQ.7 

The  system  shall  be  integrated  with  the  sighting 
system  to  acquire  threat  range,  and  the  information 
shall  be  shared  through  broadcast  in  the  tactical 
network. 

REQ.8 

The  system  design  shall  conform  to  open 
architecture  standard  with  extensible  interface  to 
integrate  with  other  external  systems. 

REQ.9 

The  system  shall  provide  operators  with  tools  to 
create,  edit,  view,  send,  and  acknowledge  tasks. 

E.  POSITED  COMMAND  AND  CONTROL  SYSTEM  SOLUTION  SET 

The  land  platforms  in  which  the  C2  system  is  installed  and  deployed  are  the 
armored  vehicles  illustrated  in  Figure  15.  The  armored  vehicle  team  forms  the  tactical 
network  to  facilitate  information  exchange  to  establish  the  common  battlefield  situation. 
This  concept  is  similar  to  the  Battlefield  Management  System  (BMS)  deployed  in 
Singapore  Armed  Force  (SAF)  (Pan  2007).  From  Figure  15,  the  commander  is  able  to 
gain  situational  awareness  in  which  he  knows  where  his  friendly  forces  are  and  what  the 
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characterization  of  the  threat  is  using  the  C2  system.  If  the  commander  wants  to  engage 
the  threat,  he  can  assign  a  task  to  his  subordinate  base  on  his  assessment  on  the  situation. 


Figure  15.  Command  and  Control  System  in  the  Land  Battlefield  Concept 


C2  System  Setup  in  the  Land  Battlefield 
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To  achieve  the  C2  capabilities  for  the  land  platform  system,  it  is  important  to 
identify  the  interaction  between  the  C2  and  the  various  subsystems  in  the  platform 
illustrated  in  Figure  16.  Each  platform  is  equipped  with  a  mission  computer,  navigation 
system,  sighting  system,  and  communication  system.  The  C2  system  interacts  with  the 
navigation  system  for  location  update  and  the  C2  system  interacts  with  the 
communication  system  to  exchange  information  among  the  combat  team.  The  C2  system 
interacts  with  the  sighting  system  for  range  information.  All  the  information  is  processed 
by  the  C2  system  to  establish  a  common  situation  picture  among  the  combat  team.  The 
C2  system  is  installed  in  the  computer,  which  connects  physically  to  the  rest  of  the 
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subsystems  (navigation  and  communication  system).  Each  land  platform  would  be 
installed  with  the  similar  configuration  so  that  during  mission,  the  commander  can 
establish  a  common  situational  picture  graphically  through  the  computer  display  via  the 
information  exchange,  such  position,  peer  location,  and  task  messages.  Each  operator’s 
call-sign  (e.g.,  commander,  subordinate  #1,  and  subordinate  #2)  in  the  combat  team  is 
used  for  identification  during  the  tactical  network  establishment. 

To  ensure  the  commander’s  tank  is  able  to  interact  with  his  subordinate  tank,  both 
C2  systems  use  the  same  messaging  protocol  with  defined  message  fonnat.  For  example, 
the  movement  of  the  subordinate  is  retrieved  by  the  subordinate  C2  system  through  its 
interaction  with  the  subordinate  navigation  system.  The  subordinate  C2  system  would 
transmit  the  location  message  to  the  commander’s  C2  system  so  that  the  commander  is 
able  to  track  his  subordinate’s  movement. 


Figure  16.  Information  Exchange 
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The  software  architecture  of  the  C2  system  is  illustrated  in  Figure  17.  The 
software  architecture  consists  of  three  layers:  presentation,  application,  and  data.  The 
presentation  layer  interacts  with  operators  through  the  graphical  user  interface  (GUI),  and 
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user  input  would  be  passed  to  the  application  layer  for  processing.  The  application  layer 
processes  the  user  input  and  retrieves  the  information  from  the  data  layer.  The  data  layer 
interacts  with  the  database  and  the  subsystems  to  collect  the  information  and  return  it  to 
the  application  layer  for  processing.  Upon  processing  completion,  the  application  layer 
returns  the  result  to  the  presentation  layer  to  present  it  to  the  operator  through  the  GUI. 


Figure  17.  Software  Architecture  of  the  Command  and  Control  System 
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F.  DETERMINE  THEORETICAL  FOUNDATION 

Command  concept  and  control  theory  are  fundamental  in  determining  the  success 
usage  of  the  deployed  C2  on  the  land  battlefield. 

1.  Command  Concept 

Builder,  Bankes,  and  Nordin  provide  the  definition  of  the  command  concept  as: 

A  vision  of  prospective  military  operations  that  informs  the  making  of 
command  decisions  during  that  operation.  (1999,  14) 
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Builder,  Bankes,  and  Nordin  go  on  and  provide  the  following  recommendation  on 
the  ideal  factors  to  be  available  in  the  command  concept: 

•  Time  scales  that  reveal  adequate  preparation  and  readiness,  not  just  of  the  concept 
but  of  the  armed  forces  tasked  with  carrying  out  that  concept. 

•  Awareness  of  the  key  physical,  geographical,  and  meteorological  features  of  the 
battle  space — situational  awareness — that  will  enable  the  concept  to  be  realized. 

•  A  structuring  of  forces  consistent  with  the  battle  tasks  to  be  accomplished. 

•  Congruence  of  the  concept  with  the  means  for  conducting  the  battle. 

•  Knowledge  of  what  can  be  accomplishment,  from  the  highest  to  the  lowest  levels 
of  command. 

•  Intelligence  on  what  the  enemy  is  expected  to  do,  including  confirming  and 
refuting  signs  to  be  looked  for  throughout  the  coming  engagement. 

•  Knowledge  of  what  the  enemy  is  trying  to  accomplish,  not  just  what  its 
capabilities  and  dispositions  may  be. 

•  Knowledge  of  what  the  concept-originating  commander  and  his  forces  should  be 
able  to  do  and  how  to  do  it,  with  all  of  the  problems  and  opportunities,  not  just  the 
required  deployments,  logistics,  and  schedules,  but  also  the  nature  of  the  clashes 
and  what  to  expect  in  the  confusion  of  battle. 

•  Indicators  of  the  failure  of,  or  flaws  in,  the  command  concept  and  ways  of 
identifying  and  communicating  infonnation  that  would  change  or  cancel  the 
concept. 

•  A  contingency  plan  in  the  event  of  failure  of  the  concept  and  the  resulting 
operation  (Builder,  Bankes,  and  Nordin  1999,  21-22). 


2.  Control  Theory 

Lawson’s  model  uses  control  theory  to  describe  the  C2  process  as: 

coordinating  the  activities  of  multiple  independent  units  and  adapting  to 
exogenous  change  in  the  battlefield  to  that  of  the  activities  performed  in 
the  control  of  the  industrial  processes  (Lawson  1981,  5). 

Control  theory,  as  illustrated  in  Figure  18,  relates  to  the  control  aspect  of  the  C2. 
Lawson’s  model  explains  how  the  commander  senses  the  situation,  through  either 
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feedback  from  subordinates  or  from  ground  assessment,  and  makes  decision  to  lead  the 
battlefield  to  the  desired  state  that  determines  the  mission  success. 


Figure  18.  Lawson’s  Model  (from  Lawson  1981,  7) 


G.  INTEGRATIVE  FRAMEWORK  FORMALIZATION 

The  research  work  in  this  thesis  shall  apply  the  cognitive  and  procedure  domain 
within  the  Langford’s  integrative  framework  to  develop  MOE  as  illustrated  in  Figure  19. 
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Figure  19.  Modified  Integrative  Framework  (from  Langford  2014,  15) 
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(MOE-i);  Influence  describing  user 
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physical  elements  (MOE-s);  Processes 
and  mechanisms  resulting  in  the 
development  of  all  physical  elements 
and  operational  contexts  (MOE-x) 

1.  Cognitive  Domain  MOE 

Langford  (2012)  describes  the  cognitive  domain  and  its  impact  on  the  product  or 
service  as  follows: 

The  cognitive  domain  involves  the  abstractions  and  reasoning  that  take 
place  when  thinking  about  a  particular  subject.  All  types  and  modes  of 
thinking  are  involved  with  the  cognitive  domain,  including 
conceptualization  and  interpretation....  For  the  cognitive  domain,  the 
relations  between  concepts  that  are  important  to  the  user  and  reflective  of 
the  user’s  intentions  should  be  represented  in  the  product  of  service 
through  the  design  of  objects,  the  enactment(s)  of  functions  that  reflect  the 
uses  as  well  as  the  decisions  that  will  be  made  with  or  as  a  consequence  of 
the  product  or  service.  (89) 
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The  cognitive  domain  MOE  consists  of  three  pairs  of  MOEs,  cognition-user, 
cognition-function  and  cognition-object.  Each  set  contains  two  MOEs  for  evaluation. 

a.  Cognition-User  MOE  (MOE-a,  MOE-p) 

The  objective  of  the  cognition-user  MOE  is  to  assess  the  value  of  the  C2  system 
to  the  end-user  in  mission  accomplishment.  In  addition,  it  can  also  provide  the  feasibility 
studies  for  new  requirement,  i.e.,  integration  with  new  subsystem.  The  two  MOEs  relate 
to  different  underlying  measures:  MOE-a  (relates  to  processes  and  activities  that  are 
measured  by  verification)  and  MOE-p  (relates  to  MOPs  to  functions  that  are 
quantifiable). 

(1)  MOE-a:  Stakeholder’s  performance  when  system  processes  are  available 
(or  not  available)  for  use 

MOE-a  characterizes  the  processes  and  activities  that  must  occur  in  order  to 
formulate  and  execute  tasks.  The  basic  question  is — are  the  requisite  processes  in  place 
and  operative?  This  question  is  verifiably,  yes  or  no.  Consider  the  three  processes:  gain 
awareness  of  the  battle  situation  (e.g.,  receive  a  mission  from  higher  command),  make  a 
decision  (part  of  which  is  to  be  decision- fit),  and  then  execute  the  decision  (e.g.,  assign  or 
reassignment  tasks). 

The  verification  is  conducted  through  mission  log  analysis  and  feedback  from  the 
user  during  an  After-Action-Review  (AAR).  MOE-a  provides  insight  into  the  C2  system 
influence  on  user  perfonnance. 

Similarly,  the  time  taken  to  make  a  decision  and  the  time  taken  to  execute  the 
decision  are  referenced  to  the 

(2)  MOE-p:  Stakeholder’s  performance  when  system  functions  are  available 
(or  not  available)  for  use 

This  set  of  MOEs  provides  a  measurement  of  the  end-user’s  cognitive  functioning 
when  three  functions  are  performed,  i.e.,  gain  awareness  of  the  battle  situation  (e.g., 
receive  a  mission  from  higher  command);  make  a  decision  (part  of  which  is  to  be 
decision-fit);  and  then  execute  the  decision  (e.g.,  assign  or  reassigmnent  tasks).  The 
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measurement  parameters,  i.e.,  the  MOPs,  are  the  time  taken  to  gain  awareness  of  the 
battle  situation,  the  time  taken  to  make  a  decision,  and  the  time  taken  to  order  execution. 


The  evaluation  is  conducted  through  analysis  of  the  user’s  mission  log  and  sensor 
performances  during  an  AAR.  MOE-p  provides  insight  into  user  performance  with  and 
without  the  C2  system  influence. 

,  „  Time  taken  from  mission  received  to  plan  and  make  decision 

MOE -  p=  - - - - - 

Time  taken  from  mission  to  order  execution  n  , 


b.  Cognition-Function  MOE  (MOE-g,  MOE-c) 

The  objective  of  the  cognition-function  MOE  is  to  assess  the  value  of  the  C2 
system  functions  in  terms  of  information  exchange.  It  evaluates  how  effective  does  the 
information  presented  to  the  user  via  the  C2  functions  contribute  to  the  situational 
awareness  and  decision-making.  For  situational  awareness,  the  C2  system  shall  monitor 
the  transaction  between  the  functions,  i.e.,  tracking  own  position  against  threat  position, 
tracking  task  expiration,  and  alert  user  of  any  incoming  threat  or  task  overdue.  For 
decision-making,  the  C2  system  shall  process  the  transaction  between  the  functions,  i.e., 
send/receive  task  assignment,  send/receive  task  status  update,  and  send/receive  position 
update,  to  enable  user  to  assess  and  respond  to  the  situation.  In  addition,  this  pair  of 
MOES  can  also  provide  the  feasibility  studies  for  new  requirement,  i.e.,  refine  the 
function.  Two  MOEs,  MOE-g  and  MOE-c  are  developed. 

(1)  MOE-g:  System  Functionality  Prognostication 

This  pair  of  MOEs  provides  a  measurement  of  the  users’  ability  to  prognosticate 
the  outcome  of  the  C2  system  function.  It  is  of  critical  importance  that  the  C2  system  is 
predictable  to  ensure  trust  established  between  the  user  and  the  C2  system.  For  instance, 
the  user  should  predict  a  warning  alert  from  the  C2  system  when  the  moving  unit  enters 
into  the  firing  sector  or  encounters  an  obstacle  during  maneuver.  The  system  warning 
alert  can  be  implemented  by  using  the  mission  monitor  module  in  the  C2  system  to 
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compare  the  unit  location  retrieved  from  the  navigation  system  against  the  existing 
danger  boundary  stored  in  the  database  system. 

The  evaluation  is  conducted  through  mission  log  analysis  and  feedback  from  user 
during  AAR.  MOE-g  evaluates  how  well  the  C2  system  functions  monitor  the  situation  in 
the  land  battlefield,  i.e.,  tracking  own  position  against  threat  position,  and  tracking  task 
expiration,  and  alert  user  of  any  incoming  threat  or  task  overdue. 

MOE-g  =  fnx  x  Wj  +  yh2  x  w2  + ...  +  fnx  x  wx  (2) 


=  £fnxxwx 

x=l 

Where  fn  is  the  C2  system  function  score  and  wx  is  the  weightage  allocated  to  the 
function.  Each  C2  system  function  score  is  allocated  1  to  5  with  1  as  the  least  desired  and 
the  weightage  is  allocated  1  to  5  with  1  as  the  least  important. 

(2)  MOE-c:  System  Functionality  Expectation 

MOE-c  verifies  that  the  C2  system  enacts  the  functions,  i.e.,  send/receive  task 
assignment,  send/receive  task  status  update,  send/receive  and  position  update,  and  present 
to  user  to  facilitate  decision-making.  The  verification  is  conducted  through  mission  log 
analysis  and  feedback  from  users  during  an  AAR.  The  C2  system  functionality 
expectation  MOE-c  identifies  user  satisfaction  based  on  system  expectation  of  having  or 
not  having  the  functions  available. 

c.  Cognition-Object  MOE  (MOE-e,  MOE-r) 

The  C2  system  is  comprised  of  several  C2  physical  objects  (e.g.,  computer, 
navigation  equipment,  display,  keyboard,  mouse,  sighting  equipment,  and  combat  radio) 
connected  to  provide  the  situational  awareness  and  decision  making  aids.  The  objective 
of  the  cognitive-object  MOE  pair  is  to  determine  the  value  of  the  physical  objects  in 
terms  of  the  users’  experience  and  the  responsiveness  to  support  the  mission.  Two  MOEs, 
MOE-e  and  MOE-r  are  developed. 
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(1)  MOE-e:  System  Experience 

The  operators  for  the  C2  systems  may  execute  actions  related  to  the  two  C2 
functions:  command  and  control.  The  MOE-e  evaluates  the  user  experience  with  the  C2 
physical  object  (computer,  navigation  equipment,  display,  keyboard,  mouse,  sighting 
equipment  and  combat  radio)  contribute  to  the  mission  accomplishment.  Interactions  with 
the  physical  objects  are  often  referred  to  as  Man-Machine  Interface  or  Human  Systems 
Integration.  If  the  user  interface  creates  inefficiency  in  operation  due  to  excessive  tree  or 
hierarchical  structures  to  navigate  accessing  data,  then  the  effectiveness  of  the  task  and  its 
impacts  on  the  mission  operations  will  be  known  through  an  evaluation.  MOE-e  deals 
with  the  human-machine  interface.  The  evaluation  is  conducted  through  mission  log 
analysis  and  feedback  from  user  during  an  AAR. 

MOE-e  =  objx  x  w,  +  obj2  x  w2  +...+ objx  x  wx  (3) 

y 

x=l 

where  objx  is  the  C2  physical  object  score  and  wx  is  the  weightage  allocated  to  the 
function.  Each  C2  physical  object  score  is  allocated  1  to  5  with  1  as  the  least  desired  and 
the  weightage  is  allocated  1  to  5  with  1  as  the  least  important. 

(2)  MOE-r:  System  Responsiveness 

The  objective  of  the  MOE-r  is  to  evaluate  the  responsiveness  of  the  C2  system 
when  the  connected  physical  C2  objects  interact  with  one  another.  Consider  an 
interaction  between  a  user  and  a  computer  interface  that  is  necessary  to  carry  out  an 
assigned  task.  The  user  inputs  the  task  instruction  into  the  display  system.  The  software 
interface  receives/  processes/  transmits  the  task  assignment  message  using  the  combat 
radio  connected  to  the  computer.  The  interaction  between  the  physical  objects  is 
evaluated  in  terms  of  their  responsiveness  and  the  low  MOE-r  indicates  the  needs  to 
review  the  interface  between  the  objects.  MOE-r  is  a  functional  measurement  of  system 
performance  in  response  to  queries  from  users.  The  evaluation  is  conducted  through 
mission  log  analysis  and  feedback  from  users  during  an  AAR. 
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MOE-r  -  objx  x  w,  +  obj2  x  w2  + ...  +  objx  x  w 


(4) 


x=l 


where  objx  is  the  C2  physical  object  score  and  wx  is  the  weightage  allocated  to  the 
function.  Each  C2  physical  object  score  is  allocated  1  to  5  with  1  as  the  least  desired  and 
the  weightage  is  allocated  1  to  5  with  1  as  the  least  important. 

2.  Procedural  Domain  MOE 

Langford  (2012,  91)  describes  the  procedural  domain  and  its  influence  over  the 
product  or  service  as  follows: 

Procedural  structures,  i.e.,  “paradox,”  take  into  account  the  stakeholder 
requirements  for  differentiating  products  and  services  from  “other” 
products  and  services.  Especially  important  for  new  product  or  service 
development  is  the  demand  to  distinguish  the  product  or  service  from 
competitive  offerings.  User  behaviors  often  reflect  the  novelty  in  products 
and  services  in  their  feelings  toward  their  work,  their  colleagues,  and  their 
involvement  in  teamness. 

The  procedural  domain  MOE  consists  of  three  set  of  MOEs,  procedure-user, 
procedure-function  and  procedure-object.  Each  pair  contains  two  MOEs  for  evaluation. 

a.  Procedure-Behavior  MOE  (MOE-i,  MOE-f) 

The  objective  of  the  procedure-user  MOE  is  to  measure  task  efficiency  of  the  C2 
system  in  terms  of  the  task  workflow.  The  procedure -user  MOE  enables  the  user  to 
determine  the  desired  workflow  towards  mission  accomplishment.  Two  MOEs,  MOE-i 
and  MOE-j  are  developed. 

(1)  MOE-i:  Influence  of  Procedures  When  System  Is  in  Use 

This  pair  of  MOEs  provides  a  measurement  on  the  individual  end-user 
performance  driven  by  the  procedures/workflow  of  executing  the  task  while  using  the  C2 
system.  The  measurement  parameter  consists  of  recording  the  steps  taken  to  gain 
awareness  of  the  battle  situation  and  to  execute  decisions  (e.g.,  task  assignment  or  task 
reassignment).  In  addition,  this  pair  of  MOEs  can  be  reused  to  evaluate  the  effectiveness 
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of  the  C2  system  for  new  requirements  (e.g.,  change  in  workflow/procedure,  and 
improving  human  system  interface). 

The  evaluation  is  conducted  through  mission  log  analysis  and  feedback  from 
users  during  an  AAR.  The  MOE-i  provides  insight  into  the  individual  end-user 
performance  driven  by  the  procedures/workflow  of  executing  the  task  while  using  the  C2 
system. 

MOE-i  =  behavior ■  x  w,  +  behavior2  x  w2  + ...+  behaviorx  x  wx  (5) 

y 

=  V  behavior  x  w 

x  x 

x\ 

where  behavior  is  the  user  score  and  wx  is  the  weightage  allocated  to  the  function.  Each 
user  score  is  allocated  1  to  5  with  1  as  the  least  desired  and  the  weightage  is  allocated  1  to 
5  with  1  as  the  least  important. 


(2)  MOE-f:  Influence  of  Procedures  When  System  Is  Not  in  Use 

This  pair  of  MOEs  provides  a  measurement  of  the  individual  end-user 
performance  driven  by  the  procedures/workflow  of  executing  the  task  without  using  the 
C2  system.  The  measurement  parameter  consists  of  recording  the  steps  taken  to  gain 
awareness  of  the  battle  situation  to  execute  decisions  (e.g.,  task  assignment  or  task 
reassignment)  and  to  realize  the  decision  outcome. 

The  evaluation  is  conducted  through  mission  log  analysis  and  feedback  from 
users  during  an  AAR.  The  MOE-f  provides  insight  into  the  individual  end-user 
performance  driven  by  the  procedures/workflow  of  executing  the  task  while  not  using  the 
C2  system. 

MOE-f  =  behavior \  x  w1  +  behavior2  x  w:  + . . .  +  behaviorx  xwx  (6) 

y 

=  2_behaviorx  x  wx 

X=1 
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where  behavior  is  the  user  score  and  wx  is  the  weightage  allocated  to  the  function.  Each 
user  score  is  allocated  1  to  5  with  1  as  the  least  desired  and  the  weightage  is  allocated  1  to 
5  with  1  as  the  least  important. 

b.  Procedure-Function  MOE  (MOE-u,  MOE-b) 

The  objective  of  the  procedure-function  MOE  is  to  evaluate  how  well  the  C2 
functions  fit  into  the  user  workflow.  Consider  the  threat  encounter  workflow.  It  is  rational 
for  commander  to  exercise  his  judgment  based  on  his  training  and  experience  to  either 
engage  the  threat  or  evacuate  in  this  critical  junction.  In  this  case,  the  voice 
communication  between  the  commander  and  his  subordinate  using  the  combat  radio 
would  be  preferred  to  using  C2  function  for  task  assignment.  However,  if  the  C2  system 
is  closely  integrated  with  subsystems  (weapon  system,  sighting  system,  communication 
system)  so  that  the  C2  functions  can  track  any  high  intensity  engagement  activities  can 
auto-update  to  higher  command  for  support  reinforcement  without  the  need  for  the 
commander  to  spend  additional  effort  to  update  the  situation,  the  C2  function  would  score 
higher  MOE  in  term  of  procedure  support.  Two  MOEs,  MOE-u,  and  MOE-b  are 
developed. 

(1)  MOE-u:  System  Functional  Resource  Utilization  Validity 

The  objective  of  the  MOE-u  is  to  evaluate  the  C2  system  function  to  support  the 
mission  workflow  in  the  land  battlefield.  For  instance,  any  C2  system  function  of  low 
utilization  during  the  mission  would  score  a  low  MOE-u.  This  indicates  there  is  a 
preferred  alternative,  i.e.,  using  voice  command  to  assign  task,  over  the  C2  system 
function.  Therefore,  MOE-u  helps  to  bridge  the  gap  between  the  user  expectation  and  the 
C2  system  developer  through  recognizing  the  actual  value  of  the  C2  system  functions 
towards  the  user’s  mission  accomplishment 

MOE-u  =  fn]  x  w,  +  fn2  x  w2  + ...  +  fnx  x  wx  (7) 


=  E/«xXWv 
X=l 
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where  fnx  is  the  C2  system  function  score  and  wx  is  the  weightage  allocated  to  the 
function.  Each  C2  system  function  score  is  allocated  1  to  5  with  1  as  the  least  desired  and 
the  weightage  is  allocated  1  to  5  with  1  as  the  least  important. 

(2)  MOE-b:  System  Functional  Boundary  Conditions 

The  objective  of  the  MOE-b  is  to  evaluate  the  C2  system  functions  in  term  of  how 
well  the  functions  interact  with  external  system.  For  instance,  the  sense-and-strike 
mission  that  involves  interaction  between  the  C2  system  function  and  the  UAV  for  the 
“sense,”  and  the  interaction  between  the  C2  system  function  and  the  artillery  system  for 
the  “strike.”  The  evaluation  is  conducted  through  mission  log  analysis  and  feedback  from 
users  during  an  AAR.  A  low  MOE-b  score  indicates  the  limitation  of  the  C2  system 
function  for  improvement. 

MOE-b  =  fn]  x  Wj  +  fn2  x  w2  + ...  +  fnx  x  w  (8) 


=  ZAXWv 

x=l 

where  fnx  is  the  C2  system  function  score  and  wx  is  the  weightage  allocated  to  the 
function.  Each  C2  system  function  score  is  allocated  1  to  5  with  1  as  the  least  desired  and 
the  weightage  is  allocated  1  to  5  with  1  as  the  least  important. 

c.  Procedure-Object  MOE  (MOE-s,  MOE-x) 

The  C2  system  is  comprised  of  several  C2  physical  objects  (computer,  navigation 
equipment,  display,  keyboard,  mouse,  sighting  equipment  and  combat  radio)  connected  to 
provide  the  situational  awareness  and  decision  making  aids.  The  objective  of  the 
procedure-object  MOE  pair  is  to  determine  the  value  of  the  physical  objects  in  tenn  of 
their  suitability  to  support  the  mission.  Consider  the  disruption  due  to  equipment  faults 
during  the  mission.  The  challenge  to  recover  the  system  and  resume  the  task  execution 
depends  on  the  spare  availability,  system  configuration,  work-around  option  and  support 
element  available.  For  instance,  users  can  execute  the  task  using  the  touch  screen  display 
if  the  keyboard  is  faulty;  users  can  change  communication  frequency  with  minimal  hassle 
if  the  main  channel  is  jammed;  there  are  spare  batteries  available  to  power  the  equipment 
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if  the  main  power  source  is  down.  In  addition,  it  can  also  provide  the  feasibility  studies 
for  object  replacement,  i.e.,  replace  the  computer  system  with  mobile  computing  device, 
and  replace  the  keyboard  with  controller.  Two  MOEs,  MOE-s  and  MOE-x  are  developed. 
Different  stakeholders  (project  manager,  system  architect,  engineers,  end-users)  can 
participate  in  the  evaluation  and  the  MOE  score  would  be  reviewed  during  stakeholders’ 
meeting. 

(1)  MOE-s:  System  Selection  Validity 

The  objective  of  the  MOE-s  is  to  validate  the  effectiveness  of  the  C2  system  in 
term  of  system  setup;  configuration,  operation,  troubleshooting  and  mission  log 
collection,  i.e.,  to  question  the  support  effort  for  the  use  of  the  C2  system.  For  instance, 
consider  the  procedure  and  effort  required  to  replace  the  faulty  item  during  the  mission 
would  determine  its  effectiveness  to  the  overall  C2  system. 

y 

MOE-s  =  ^  objx  x  wx  (9) 

X=1 

where  objx  is  the  C2  physical  object  score  and  wx  is  the  weightage  allocated  to  the 
function.  Each  C2  physical  object  score  is  allocated  1  to  5  with  1  as  the  least  desired  and 
the  weightage  is  allocated  1  to  5  with  1  as  the  least  important. 

(2)  MOE-x:  System  Operational  Context  Validity 

Figure  20  and  Figure  21  illustrate  the  C2  system  interaction  for  external  systems 
and  internal  systems,  respectively. 
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Figure  20.  External  Physical  Context 


Figure  2 1 .  Internal  Physical  Context  Diagram 
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The  external  interaction  consists  of  the  communication  links  between  the 
commander  C2  system  and  the  subordinate  C2  system  and  the  higher  command  C2 
system  and  the  commander  C2  system.  The  data  exchange  (task  assignment,  task  status, 
position  update)  is  facilitated  via  connection  link  between  the  stations.  The  internal 
interaction  contains  the  links  between  the  C2  system  and  subsystems  in  the  land  platform. 
The  objective  of  the  MOE-x  is  to  evaluate  the  C2  system  interaction  with  other  systems 
so  that  limitation  can  be  identified  and  improved. 

MOE-e  =objx  x  w,  +obj2  x  w2  +...+objx  x  wx  (10) 

y 

x=l 

where  objx  is  the  C2  physical  object  score  and  wx  is  the  weightage  allocated  to  the 
function.  Each  C2  physical  object  score  is  allocated  1  to  5  with  1  as  the  least  desired  and 
the  weightage  is  allocated  1  to  5  with  1  as  the  least  important. 
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H.  ANALYZE  AND  EVALUATE  EFFECTIVENESS 


The  12  pairs  of  MOEs  as  tabulated  in  Table  7  are  used  to  evaluate  the 
effectiveness  of  using  the  C2  system  used  in  annor  combat  team  as  illustrated  in  Figure 
25.  Both  cognition  and  procedure  domain  MOEs  are  calculated  and  the  result  are 
tabulated.  The  system  to  be  evaluated  in  the  C2  system  deployed  on  the  land  platform, 
taking  reference  from  the  Battlefield  Management  System  (BMS)  deployed  in  Singapore 
Armed  Force  (SAF)  (Pan  2007). 


Table  7.  List  of  Measures  of  Effectiveness  to  Evaluate  the  Command  and 

Control  System  in  the  Land  Battlefield 


Domain 

MOE 

Description 

Cognition-User 

MOE-a 

Stakeholder's  performance  when  system  in  use 

MOE-p 

Stakeholder's  performance  when  system  not  in  use 

Cognition-Function 

MOE-g 

System  Functionality  Prediction 

MOE-c 

System  Functionality  Expectation 

Cognition-Object 

MOE-e 

System  Experience 

MOE-r 

System  Responsiveness 

Procedure-Behavior 

MOE-i 

Influence  on  procedure  when  system  in  use 

MOE-f 

Influence  on  procedure  when  system  in  not  in  use 

Procedure-Function 

MOE-u 

System  Functional  Resource  Utilization  Validity 

MOE-b 

System  Functional  Boundary  Conditions 

Procedure-Object 

MOE-s 

System  Selection  Validity 

MOE-x 

System  Operational  Context  Validity 
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Figure  22.  C2  System  MOE  Evaluation 
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Cognition-Object  (MOE-e  /  MOE-r) 

System  Experience  /  Responsiveness. 

Procedure-Object  (MOE-s  /  MOE-x) 

System  Selection  Validity  /  Operational 
Context  Validity. 


(MOE-a/MOE-p) 

Measure  the  mission  effectiveness 
_  with  or  without  the  C2  system. 
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1.  Cognition  Domain  MOE  Calculation 

To  calculate  the  MOE  at  the  cognition  domain,  the  scenario  is  illustrated  in  Figure 
26.  Consider  the  C2  systems  in  the  armor  combat  team  are  connected  in  the  tactical 
network  and  the  operators  are  using  the  C2  system. 
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Figure  23.  Cognition  Domain  MOE  Evaluation 


Cognition-Object  (MOE-g,  MOE-c) 
•  Measure  System  Experience  / 
Responsiveness 


Cognition-Function  (MOE-g,  MOE-c) 
•  Measure  C2  Function 

Effectiveness  to  support  task 
execution 


Task  Creation 

(1)  HQ  Command 

(2)  Situation 
Awareness 


T^> 


Cognition-Behavior  (MOE-a,  MOE-p) 

•  Measure  Operator  Performance 

•  Measure  Operator  Performance  without  the 
system  to  identify  the  value  of  the  C2  system 


Commander  assess  the 
situation  and  assign 
task  to  take  out  threat 


Time 

- ► 


60 


a.  Calculate  Cognition-User  MOEs  (MOE-a  / MOE-p) 

Let  the  time  taken  for  the  commander  to  receive  order  from  higher  HQ  and 
proceed  to  plan,  assign,  monitor  and  accomplish  task  be  xM0E-a5  the  time  frame  for  the 
task  to  complete  from  order  receipt  be  yMOE-a-  The  MOE-a  is  calculated  as 

MOE-a  = 

y MOE-a 

For  MOE-p  calculation,  consider  the  C2  systems  are  not  in  use  and  the 
commander  would  rely  on  voice  communication  using  the  combat  radio  to  receive  order 
from  higher  HQ,  perform  situational  awareness,  task  planning,  assignment  and 
monitoring.  Let  the  time  taken  for  the  commander  to  receive  order  from  higher  HQ  and 
proceed  to  plan,  assign,  monitor  and  accomplish  task  be  xMoe-p,  the  timeframe  for  the  task 
to  complete  from  order  receipt  be  yvioE-p-  MOE-p  is  calculated  as  follows: 

moe-p  =  x^il 

y MOE-p 

Given  that  each  task  has  a  different  timeframe  and  the  execution  time  to  complete 
the  task  tends  to  vary  hence  it  is  more  useful  to  calculate  this  pair  of  MOEs  based  on  per 
task  basis.  However,  it  is  desirable  that  the  overall  MOE-a  value  is  lesser  than  MOE-p  to 
indicate  the  operator  would  perform  better  in  their  mission  execution  with  the  use  of  C2 
system.  In  addition,  this  pair  of  MOE  can  be  repeated  to  evaluate  new  requirement  to 
manage  expectation. 

b.  Calculate  Cognition-Function  MOEs  (MOE-g  /  MOE-c) 

MOE-g  provides  a  measurement  on  the  end-users’  ability  to  prognosticate  the  C2 
system  function  predictability.  It  is  of  critical  importance  that  the  C2  system  is 
predictable  to  ensure  trust  established  between  the  user  and  the  C2  system.  The  MOE-g  is 
calculated  as  shown  in  Table  8.  Four  scenarios  are  chosen  namely,  threat  within  range, 
poor  communication  coverage,  task  incompetency  and  poor  navigation  signal  to  evaluate 
the  system  functionality  prediction,  MOE-g  of  the  system.  The  objective  is  to  evaluate 
the  effectiveness  of  the  C2  system  on  reaction  to  situation,  i.e.,  how  fast  /accurate  does 
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the  C2  system  detect/manage  the  threat.  Different  stakeholders  (project  manager,  system 
architect,  engineers,  and  end-users)  can  participate  in  the  evaluation  and  the  overall 
MOE-g  score  would  be  reviewed  during  stakeholders’  meeting. 


Table  8.  System  Functionality  Prediction  MOE-g 


S/N 

Scenario 

System  Function 
Action 

Aggregated 
Score  (1-Poor, 
5-Good) 

Weight 

(T -Non-Priority, 
5 -Priority) 

MOE-g 

1 

Threat  Within 
Range 

Sensory  function 
and  monitoring 
function  trigger 
system  alert 

3 

5 

15 

2 

Poor 

Communication 

Coverage 

Communication 
sensory  function 
trigger  system  alert 

3 

5 

15 

3 

Task  Incompetency 
(e.g.,  low 
ammunition,  fuel, 
and  logistics) 

Platform  sensory 
function  triggers 
system  alert 

3 

4 

12 

4 

Poor  Navigation 
Signal 

Navigation  sensory 
function  trigger 
system  alert 

4 

4 

16 

Total 

58 

From  the  score,  MOE-g  can  be  improved  through  implementation  of  data  analytic 
mechanism  to  enhance  the  prediction,  i.e.,  terrain  /  high  canopy  area  to  pre-empt  poor 
communication  and  navigation  signal. 

The  C2  system  functionality  expectation  MOE-c  identifies  user  satisfaction  based 
on  system  expectation  is  tabulated  in  Table  9.  Five  functions  are  chosen  namely,  task 
assignment,  allocate  resource,  analyze  risk,  and  poor  navigation  signal  to  evaluate  the 
system  functionality  expectation,  MOE-g  of  the  system.  The  objective  is  to  evaluate  the 
effectiveness  of  the  C2  function  in  mission  accomplishment,  i.e.,  how  much  does  the  C2 
function  contributes  to  command  and  control.  Different  stakeholders  (project  manager, 
system  architect,  engineers,  end-users)  can  participate  in  the  evaluation  and  the  overall 

MOE-c  score  would  be  reviewed  during  stakeholders’  meeting. 
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Table  9.  System  Functionality  Expectation  MOE-c 


S/N 

Function 

Expectation 

Aggregated  Score 
(1-Poor,  5-Good) 

Weight 
(1 -Non- 
Priority,  5- 
Priority) 

MOE-c 

1 

Task 

Assignment 

Expect  system  to 
indicate  the  task 
assignment  message  is 
sent  and  received. 

Expect  system  to 
indicate  the  task 
progress  status  based  on 
assignee’s  task 
feedback. 

Expect  system  to  alert 
operator  on  task  expiry 
or  incomplete  task. 

3 

5 

15 

2 

Allocate 

Resource 

Expect  system  to 
provide  list  of  available 
resource  (e.g.,  terrain 
information,  threat 
analysis,  and  support 
element) 

4 

5 

20 

3 

Analyze 

Risk 

Expect  system  to  update 
situation  picture 
especially  threat  at 
regular  interval. 

4 

3 

12 

4 

Negotiate 

Constraint 

Expect  system  to  update 
situation  picture 
especially  remaining 
resources  at  regular 
interval. 

4 

3 

12 

5 

Report 

Expect  system  to 
indicate  the  task 
assignment  message  is 
sent  and  received. 

3 

5 

15 

Total 

69 

One  challenge  faced  to  achieve  high  MOE-c  score  is  that  the  terrain  on  the  land 
battlefield  constantly  affects  the  network  performance  that  often  led  to  dropped  C2 
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messages  or  late  C2  messages.  While  there  are  recommendations  to  improve  the  network 
performance  such  as  smaller  message  size,  rebroadcast  station  deployment  and  instilling 
network  discipline,  communication  in  the  land  battlefield  still  remains  an  operational 
challenge. 


c.  Calculate  Cognition-Object  MOEs  (MOE-  e  /MOE-r) 

MOE-e  provides  insight  into  the  effectiveness  of  system  usage  related  to 
command  and  control.  The  MOE-e  is  calculated  as  shown  in  Table  10.  The  C2  system 
comprises  of  several  physical  objects  connected  together  to  provide  the  situational 
awareness  and  decision  making  tool  for  the  operators.  The  objective  of  the  MOE-e  is  to 
determine  whether  the  physical  object  is  the  right  tool  for  the  operation,  i.e.,  to  question 
the  effectiveness  of  the  physical  object  to  mission  accomplishment.  Different 
stakeholders  (project  manager,  system  architect,  engineers,  and  end-users)  can  participate 
in  the  evaluation  and  the  overall  MOE-e  score  would  be  reviewed  during  stakeholders’ 
meeting.  Based  on  the  result,  there  is  a  need  to  improve  the  score  of  the  human  interface 
by  looking  at  the  different  alternative  of  human  interface  (touch  screen,  application  input 
form,  and  button  layout). 


Table  10.  System  Experience  MOE-e 


S/N 

C2  Physical 
Object 

Usage  Description 

Experience 
Score 
(1-Poor,  5- 
Good) 

Weight 
(1 -Non- 
Priority,  5- 
Priority) 

MOE-e 

1 

Display  system 

Display  situation  picture 
Provides  Graphical  User 
Interface  (GUI) 

4 

5 

20 

2 

Computer 

system 

Processes  the  C2  Function 

3 

2 

6 

3 

Human 

Interface 

Provides  interface  for 
operator  input 

2 

5 

10 

64 


S/N 

C2  Physical 
Object 

Usage  Description 

Experience 
Score 
(1-Poor,  5- 
Good) 

Weight 
(1 -Non- 
Priority,  5- 
Priority) 

MOE-e 

4 

C2  Software 

Provides  C2  Function 

3 

5 

15 

5 

Sighting 

system 

Provides  interface  for 
operator  to  measure  range 

3 

3 

9 

6 

Comm.  System 

Provides  tactical 
communication. 

3 

3 

9 

7 

Navigation 

System 

Provides  position  update 

5 

2 

10 

8 

Network 

System 

Connects  the  combat  team 

2 

3 

6 

Total 

85 

The  C2  system  response  MOE-r  provides  insight  into  the  effectiveness  of  system 
usage  related  to  command  and  control.  The  MOE-r  is  calculated  as  shown  in  Table  11. 
The  C2  system  is  comprised  of  several  physical  objects  connected  together  to  provide  the 
situational  awareness  and  decision  making  tool  for  the  operators.  The  objective  of  the 
MOE-r  is  to  determine  whether  the  right  tool  is  operating  effectively,  i.e.,  to  question  on 
the  physical  object’s  specification  to  mission  accomplishment.  Different  stakeholders 
(project  manager,  system  architect,  engineers,  end-users)  can  participate  in  the  evaluation 
and  the  overall  MOE-r  score  would  be  reviewed  during  stakeholders’  meeting.  Based  on 
the  result,  there  is  a  need  to  improve  the  score  of  the  network  system  by  looking  at  the 
network  configuration  (number  of  stations  in  a  network,  allowable  message  size,  and 
information  exchange  schema). 
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Table  11.  System  Responsiveness  MOE-e 


S/N 

C2  Object 

Usage  Description 

Responsive 
Score 
(1-Poor,  5- 
Good) 

Weight 
(1 -Non- 
Priority,  5- 
Priority) 

MOE-r 

1 

Display  system 

Display  situation  picture 
Provides  Graphical  User 
Interface  (GUI) 

4 

5 

20 

2 

Computer 

System 

Processes  the  C2  Function 

3 

5 

15 

3 

Human 

Interface 

Provides  interface  for 
operator  input 

5 

5 

25 

4 

C2  Software 

Provides  C2  Function 

3 

5 

15 

5 

Sighting 

System 

Provides  interface  for 
operator  to  measure  range 

3 

5 

15 

6 

Comm.  System 

Provides  tactical 
communication. 

3 

5 

15 

7 

Navigation 

System 

Provides  position  update 

5 

5 

25 

8 

Network 

System 

Connects  the  combat  team 

2 

5 

10 

Total 

140 

One  challenge  to  achieve  high  MOE-e  and  MOE-r  scores  lies  in  the  system 
integration.  Langford  (2012)  described,  “The  more  ambitious  the  integration,  and  the 
more  out  of  control  are  the  interfaces,  i.e.,  not  under  change  control  or  management,  the 
more  difficult  the  integration  of  the  new  product  or  service  into  the  existing  users’ 
environment  and  enterprise.”  Therefore,  the  system  integrator  plays  an  important  role  to 
work  with  the  different  stakeholders  to  ensure  the  C2  objects  are  integrated  into  the  C2 
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systems  of  better  user  experience  and  responsiveness.  In  addition,  there  is  a  need  to  start 
integration  early  and  evaluate  the  MOEs  with  each  integrated  build  so  that  any  potential 
risk  can  be  mitigated  and  resolved. 


2.  Procedure  Domain  MOE  Calculation 

To  calculate  the  MOE  at  the  procedure  domain,  the  scenario  is  illustrated  in 
Figure  27.  Consider  the  C2  systems  in  the  annor  combat  team  are  connected  in  the 
tactical  network  and  the  operators  are  using  the  C2  system.  The  C2  system  is  comprised 
of  several  physical  objects  connected  together  to  provide  the  situational  awareness  and 
decision  making  tool  for  the  operators.  The  objective  of  the  MOE-r  is  to  detennine 
whether  the  right  tool  is  operating  effectively,  i.e.,  to  question  on  the  physical  object’s 
specification  to  mission  accomplishment.  Different  stakeholders  (project  manager, 
system  architect,  engineers,  end-users)  can  participate  in  the  evaluation  and  the  overall 
MOE-r  score  would  be  reviewed  during  stakeholders’  meeting.  Based  on  the  result,  there 
is  a  need  to  improve  the  score  of  the  network  system  by  looking  at  the  network 
configuration  (number  of  stations  in  a  network,  allowable  message  size,  and  information 
exchange  schema). 
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Figure  24.  Procedure  Domain  MOE  Evaluation 
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a.  Calculate  Procedure-Behavior  MOEs  (MOE-  i /MOE-  f) 

The  MOE-i  provides  insight  into  the  individual  end-user  performance  driven  by 
the  procedures/workflow  of  executing  the  task  while  using  the  C2  system.  The  MOE-i  is 
calculated  as  shown  in  Table  12.  The  C2  procedure  is  comprised  of  several  activities  such 
as  task  planning,  task  assignment,  task  monitoring  and  recovery.  The  objective  of  the 
MOE-i  is  to  determine  how  effective  the  operator  accomplishes  the  mission  with  the  use 
of  the  C2  system,  i.e.,  to  quantify  the  value  by  the  C2  system  /  add-on  requirement. 
Different  stakeholders  (project  manager,  system  architect,  engineers,  and  end-users)  can 
participate  in  the  evaluation  and  the  overall  MOE-i  score  would  be  reviewed  during 
stakeholders’  meeting. 
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Table  12.  Influence  of  Procedures  When  System  is  in  use  MOE-i 


S/N 

C2  Procedures 

Aggregated 
Score  (1-Poor, 
5-Good) 

Weight  (1- 
Non-Priority, 

5 -Priority) 

MOE-i 

1 

Perform  Task  Planning 

4 

5 

20 

2 

Perform  Task  Assignment 

4 

5 

20 

3 

Perform  Task  Acknowledge 

4 

3 

12 

4 

Perform  Task  Monitoring 

4 

5 

20 

5 

Perform  Task  Update 

4 

5 

20 

6 

Perform  Recovery  Command 

4 

3 

12 

7 

Perform  Recovery 

4 

3 

12 

Total 

116 

MOE-f  provides  a  measurement  of  the  individual  end-user  performance  driven  by 
the  procedures/workflow  of  executing  the  task  without  using  the  C2  system.  The  C2 
procedure  is  comprised  of  several  activities  such  as  task  planning,  task  assignment,  task 
monitoring  and  recovery.  The  objective  of  the  MOE-j  is  to  determine  how  effective  the 
operator  accomplishes  the  mission  without  the  use  of  the  C2  system.  Different 
stakeholders  (project  manager,  system  architect,  engineers,  and  end-users)  can  participate 
in  the  evaluation  and  the  overall  MOE-i  score  would  be  reviewed  during  stakeholders’ 
meeting.  The  intention  is  to  evaluate  the  value  of  C2  system  by  analyzing  the  difference 
between  MOE-i  and  MOE-f.  The  MOE-f  is  calculated  as  shown  in  Table  13. 
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Table  13.  Influence  of  Procedures  When  System  is  not  in  use  MOE-f 


S/N 

C2  Procedures 

Aggregated 
Score  (1-Poor, 
5-Good) 

Weight  (1- 
Non-Priority, 

5 -Priority) 

MOE-f 

1 

Perform  Task  Planning 

2 

5 

10 

2 

Perform  Task  Assignment 

2 

5 

10 

3 

Perform  Task  Acknowledge 

4 

3 

12 

4 

Perform  Task  Monitoring 

2 

5 

10 

5 

Perform  Task  Update 

4 

5 

20 

6 

Perform  Recovery  Command 

4 

3 

12 

7 

Perform  Recovery 

4 

3 

12 

Total 

86 

Both  MOE-i  and  MOE-f  score  are  highly  influenced  by  factors  such  as  leadership, 
amount  of  training  and  competency.  However,  it  is  desirable  that  MOE-i  scores  higher 
than  MOE-f.  In  addition,  this  pair  of  MOEs  can  be  repeated  to  evaluate  new  requirement 
to  manage  expectation. 

b.  Calculate  Procedure-Function  MOEs  (MOE-  u  /MOE-  b) 

MOE-u  provides  a  metric  that  measures  how  well  the  C2  functions  support  the  C2 
procedure.  A  low  MOE  score  indicates  the  need  to  re-assess  the  system  function  so  that 
procedure  can  be  improved.  The  C2  procedure  is  comprised  of  several  activities  such  as 
task  planning,  task  assignment,  task  monitoring  and  recovery.  The  objective  of  the  MOE- 
u  is  to  determine  how  effective  the  C2  function  supports  the  C2  procedure,  i.e.,  is  it  worth 
paying  for  the  C2  function  to  support  the  procedure.  Different  stakeholders  (project 
manager,  system  architect,  engineers,  and  end-users)  can  participate  in  the  evaluation  and 
the  overall  MOE-u  score  would  be  reviewed  during  stakeholders’  meeting.  The  MOE-u  is 
calculated  as  shown  in  Table  14. 
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Table  14.  System  Functional  Resource  Utilization  Validity  MOE-u 


S/N 

C2  Procedures 

Supported  by  C2 
Function 

Utilization  (1- 
Poor, 

5 -Good) 

Weight 

(1 -Non-Priority, 

5 -Priority) 

MOE-u 

1 

Perfonn  Task 
Planning 

Allocate  Resource 

4 

5 

20 

2 

Perfonn  Task 
Assignment 

Task  Assignment 

4 

5 

20 

3 

Perfonn  Task 
Acknowledge 

Task 

Acknowledgement 

4 

3 

12 

4 

Perfonn  Task 
Monitoring 

Monitor  Performance 

4 

5 

20 

5 

Perfonn  Task  Update 

Report 

4 

5 

20 

6 

Perfonn  Recovery 
Command 

Task  Assignment 

4 

3 

12 

7 

Perfonn  Recovery 

Monitor  Performance 

4 

3 

12 

Total 

116 

MOE-b  evaluates  the  C2  system  functions  in  term  of  how  well  the  functions 
interact  with  external  system.  A  low  MOE  score  indicates  the  need  to  re-assess  the 
interactions  between  the  system’s  functions  and  external  systems.  The  MOE-b  is 
calculated  as  shown  in  Table  15.  Based  on  the  result,  the  handheld  system  does  not 
interact  well  with  the  C2  functions.  Therefore,  there  is  a  need  to  review  the  workflow  of 
using  handheld  system,  i.e.,  connection  to  C2  system,  synchronization  between  handheld 
system  and  computer  system,  and  function  accessibility,  to  interact  with  the  C2  functions. 
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Table  15.  System  Functional  Boundary  Conditions  MOE-b 


S/N 

External 

System 

Interaction  with  C2  Functions 

Boundary 
Condition 
Score  (1- 
Poor, 

5 -Good) 

Weight 
(1 -Non- 
Priority, 

5 -Priority) 

MOE-b 

1 

Higher  HQ 
C2  System 

Higher  HQ  assigns  mission 
order  to  commander  using  the 
C2  Functions. 

Exchange  situation  picture 
through  the  common  C2 
Functions  and  C2  Messages 

4 

5 

20 

2 

Long  Range 
Sensor 
System  (e.g. 
UAV) 

Provides  intelligence 
information  as  input  argument 
to  C2  functions 

3 

3 

9 

3 

Support 

System 

Exchange  situation  picture 
through  the  common  C2 
Functions  and  C2  Messages 

3 

3 

9 

4 

Handheld 

System 

Exchange  situation  picture 
through  the  common  C2 
Functions  and  C2  Messages 

2 

2 

4 

Total 

42 

It  is  desirable  to  achieve  high  MOE-u  score  to  ensure  that  the  C2  functions  are 
fully  utilized  to  support  the  C2  process.  From  the  MOE-b  score,  further  enhancement  can 
be  implemented  to  improve  the  integration  with  external  system,  subject  to  operational 
need.  These  MOE  pairs  are  repeatable  to  evaluate  future  C2  system  development. 
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c.  Calculate  Procedure-Object  MOEs  (M OE-  s  /MOE-  x) 

The  objective  of  the  MOE-s  is  to  validate  the  effectiveness  of  the  C2  system  in 
term  of  system  setup;  configuration,  operation,  troubleshooting  and  mission  log 
collection,  i.e.,  to  question  the  support  effort  for  the  use  of  the  C2  system.  MOE-s  is 
calculated  as  shown  in  Table  16.  Different  stakeholders  (project  manager,  system 
architect,  engineers,  and  end-users)  can  participate  in  the  evaluation  and  the  overall 
MOE-s  score  would  be  reviewed  during  stakeholders’  meeting. 


Table  16.  System  Selection  Validity  MOE-s 


S/N 

C2  Object 

Setup  Arrangement 

Selection 
Score  (1- 
Poor, 

5 -Good) 

Weight 
(1 -Non- 
Priority, 

5 -Priority) 

MOE-s 

1 

Computer 

System 

Connected  to  LAN  with  the 
rest  of  the  C2  Objects. 

4 

5 

20 

2 

Database 

System 

Installed  in  the  Computer 
System 

4 

4 

16 

3 

Display  System 

Connected  to  the  Sighting 
and  Computer  System 

4 

5 

20 

4 

Navigation 

System 

Connected  to  the  Computer 
System  in  the  LAN. 

4 

3 

12 

5 

Sighting 

System 

Connected  to  the  Computer 
System  in  the  LAN. 

4 

3 

12 

6 

Communication 

System 

Connected  to  the  Computer 
System  in  the  LAN. 

4 

5 

20 

7 

C2  Software 

Installed  in  the  Computer 
System 

4 

5 

20 

Total 

120 
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The  objective  of  the  MOE-x  is  to  evaluate  the  C2  system  interaction  with  other 
systems  so  that  limitation  can  be  identified  and  improved.  The  MOE-x  is  calculated  as 
shown  in  Table  17.  Different  stakeholders  (project  manager,  system  architect,  engineers, 
and  end-users)  can  participate  in  the  evaluation  and  the  overall  MOE-x  score  would  be 
reviewed  during  stakeholders’  meeting. 


Table  17.  System  Operational  Context  Validity  MOE-x 


S/N 

External 

Object 

Interaction  with  C2  Functions 

System 

Operational 

Context 

(1-Poor, 

5 -Good) 

Weight 
(1 -Non- 
Priority, 

5 -Priority) 

MOE-x 

1 

Computer 

System 

Processes  the  C2  functions. 

4 

5 

20 

2 

Database 

System 

Records  the  C2  functions 
transaction 

4 

4 

16 

3 

Display 

System 

Provides  user  interface  to 
input  operator’s  argument  to 
the  C2  functions 

4 

5 

20 

4 

Navigation 

System 

Provides  navigation 

information  as  input  argument 
to  the  C2  functions  (report 
location) 

4 

3 

12 

5 

Sighting 

System 

Provides  laser  range 

information  as  input  argument 
to  the  C2  functions  (report 
threat  range). 

4 

3 

12 

6 

Comm. 

System 

Sends  and  receives  C2 
messages  generated  from  C2 
functions 

4 

5 

20 

Total 

120 
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It  is  desirable  to  achieve  high  MOE-s  and  MOE-x  score  to  ensure  that  the  C2 
objects  are  integrated  to  support  the  C2  process.  The  scores  are  subjected  to  change  due 
to  user  proficiency,  and  equipment  wear  and  tear.  These  MOE  pairs  are  repeatable  to 
evaluate  the  effectiveness  of  integration  with  new  system. 

This  chapter  covers  the  nine-step  methods  to  evaluate  the  effectiveness  of  using  a 
deployed  C2  system  in  the  land  battlefield.  The  integrative  framework  has  proven  to  be 
useful  to  develop  the  MOEs  that  are  meaningful  and  repeatable  to  evaluate  an  existing  C2 
system.  The  next  chapter  shall  conclude  the  research  work  and  further  research. 
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VI.  CONCLUSION 


This  research  provides  a  learning  opportunity  to  develop  the  deeper  understanding 
in  the  C2  system  and  various  MOE  development  processes  and  methods.  In  addition,  this 
research  serves  to  emphasis  the  importance  of  developing  meaningful  MOEs  to  evaluate 
the  effectiveness  of  using  the  deployed  C2  system  in  the  land  battlefield.  The  MOEs  can 
be  reused  to  evaluate  new  requirement  to  access  the  feasibility  and  better  manage  user 
expectation. 

(1)  Major  Findings 

To  successfully  develop  12  pairs  of  measures  of  effectiveness  (MOEs)  with  the 
integrative  framework,  the  nine-step  methodology  is  applied.  The  integrative  framework 
provides  the  comprehensive  guideline  to  develop  the  MOE  with  objective  values  and 
subjective  criteria.  The  nine-step  method’s  repeatability  facilitates  the  evaluation  of  the 
effectiveness  of  the  C2  for  each  system  refinement  (firmware/software/system  upgrade) 
or  new  requirements  (additional  of  sensor/communication  system/protocol).  Each 
evaluation  should  indicate  opportunities  for  the  system,  process,  and  organization  to 
improve. 

The  developed  set  of  C2  MOEs  allows  stakeholders  to  evaluate  the  system,  and 
the  use  of  the  integrative  framework  to  produce  repeatable  MOEs  provides  cost-savings 
opportunities  for  system  refinement  and  a  new  iteration  of  system  development. 

(2)  Further  Research 

Further  research  can  deploy  the  12  pairs  of  MOEs  to  evaluate  the  use  of  deployed 
C2  systems  in  different  land  platforms  (e.g.,  command  post,  mobile  platform,  and 
handheld  C2  systems). 

(3)  Conclusion 

The  end-user  (the  Army)  need  for  the  C2  system  to  provide  them  with  situational 
awareness  and  decision  making  aids  is  identified  as  the  most  important  factor  in  their 
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expectation  of  the  system.  As  the  expectation  of  the  stakeholder  to  stay  connected  in  the 
battlefield  for  better  situational  awareness  and  decision-making,  there  is  an  increasing 
requirement  to  integrate  faster  and  better  subsystems  (sensor,  communication  device  and 
processor)  to  the  C2  system.  However,  there  is  a  need  to  recognize  the  limitation  and 
constraint  on  the  land  battlefield  and  also  to  maintain  the  safety  aspect  of  the  system  so  as 
to  provide  a  realistic  assessment  on  the  feasibility  of  the  requirements.  To  manage 
stakeholders’  expectation,  there  is  a  need  to  evaluate  the  effectiveness  of  the  deployed  C2 
system  having  implemented  with  the  new  requirements.  The  measures  of  effectiveness 
(MOEs)  of  using  the  deployed  C2  system  are  developed  using  an  integrative  framework 
to  evaluate  the  system’s  fitness-for-purpose. 

The  key  functions  of  “To  Command”  and  “To  Control”  are  decomposed  and 
analyzed.  The  MOE  development  was  achieved  by  the  decomposition  and  analysis  of  the 
C2  function  with  respect  to  the  user  expectation.  The  boundaries  are  described  through 
physical,  functional,  and  behavioral  domains  (Langford  2012).  The  problem  domain  lies 
within  these  three  types  of  boundaries.  For  the  C2  system,  the  boundaries  extend  beyond 
the  developed  system  to  the  domain(s)  of  the  adversary.  The  MOEs  extend  to  the 
boundaries  and  encapsulate  all  functions  and  solutions  (Langford  2012). 

A  key  aspect  in  determining  effectiveness  of  a  C2  system  is  based  on  the  system 
architecture.  The  life  cycle  phases  of  the  C2  architecture  are  first  driven  by  the 
stakeholders’  needs.  Stakeholders’  perspective  should  reflect  their  individual  needs,  that 
when  combined  form  the  operating  a  C2  system.  Measures  of  effectiveness  (MOEs)  and 
functional  measures  of  perfonnance  (MOPs)  are  evaluated  and  given  to  the  program 
manager,  as  feedback  to  ensure  compliance  with  stakeholder  needs  to  ensure  that  the  C2 
system  performs  to  its  desired  capability. 

The  measures  of  effectiveness  (MOEs)  of  C2  systems  on  land  battlefields  are 
developed  and  evaluated  using  the  nine-step  methodology.  The  nine-step  methodology 
was  developed  to  unify  the  concepts  of  MOEs  into  a  repeatable,  validated  process  and  to 
identity  MOEs  associated  with  both  development  and  operations  (Langford  2014,  10). 
Using  the  nine-step  methodology  and  the  integrative  framework,  12  pairs  of  MOEs  are 
developed. 
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The  nine-step  method’s  repeatability  facilitates  the  evaluation  of  the  effectiveness 
of  the  C2  for  each  system  refinement  (firmware/software/system  upgrade)  or  new 
requirements  (additional  of  sensor/communication  system/protocol).  Each  evaluation 
should  indicate  opportunities  for  the  system,  process,  and  organization  to  improve.  The 
developed  set  of  C2  MOEs  allows  stakeholders  to  evaluate  the  system,  and  the  use  of  the 
integrative  framework  to  produce  repeatable  MOEs  provides  cost-savings  opportunities 
for  system  refinement  and  a  new  iteration  of  system  development. 

This  thesis  has  fulfilled  the  research  objectives  and  this  research  has  provided 
learning  opportunities  to  develop  meaningful  MOEs  that  evaluate  the  use  of  deployed  C2 
system  in  the  land  battlefield. 
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